ECS Virtual shutdown / restart question

Holger,

There isn't a necessary "safe" procedure to shut down ECS Community Edition; sounds like an issue with your install. Does it appear to be timing out or rejecting your credentials fairly immediately, and is this on a specific installation or both?

Within the container, you can check for login issues in /opt/storageos/logs/authsvc.log; depending on what you find, we can determine how to approach troubleshooting.

- Aaron

HI Aaron

In /opt there are no subdirectories.

In /var/log/vipr/emcvipr-object/authsvc.log the tail says:

[root@ecssinglenode /]# tail /var/log/vipr/emcvipr-object/authsvc.log

2016-04-29 07:50:08,171 [TaskScheduler-AuthService-DefaultScheduler-004]  INFO  TaskScheduler.java (line 437) Thread stats - total threads in application 248, total tasks 0

2016-04-29 07:50:14,460 [ZkConnection-17]  INFO  ZkConnection.java (line 216) check for changes in /data/vnest/vnest-main/configuration/coord-client

2016-04-29 07:50:18,172 [TaskScheduler-AuthService-DefaultScheduler-004]  INFO  TaskScheduler.java (line 420) Thread stats: shared scheduler executor: active thread 1,pool size 10, queued tasks 0; shared parallel executor: active thread 0,pool size 200, queued tasks 0

2016-04-29 07:50:18,172 [TaskScheduler-AuthService-DefaultScheduler-004]  INFO  TaskScheduler.java (line 430) Thread stats - total in thread pools: the active thread 1, pool size 210, queued tasks 0

2016-04-29 07:50:18,172 [TaskScheduler-AuthService-DefaultScheduler-004]  INFO  TaskScheduler.java (line 437) Thread stats - total threads in application 248, total tasks 0

2016-04-29 07:50:18,243 [client-heart-beat-monitor-000]  INFO  ClientHeartBeatImpl.java (line 66) start hb run

2016-04-29 07:50:24,460 [ZkConnection-17]  INFO  ZkConnection.java (line 216) check for changes in /data/vnest/vnest-main/configuration/coord-client

2016-04-29 07:50:28,172 [TaskScheduler-AuthService-DefaultScheduler-004]  INFO  TaskScheduler.java (line 420) Thread stats: shared scheduler executor: active thread 1,pool size 10, queued tasks 0; shared parallel executor: active thread 0,pool size 200, queued tasks 0

2016-04-29 07:50:28,172 [TaskScheduler-AuthService-DefaultScheduler-004]  INFO  TaskScheduler.java (line 430) Thread stats - total in thread pools: the active thread 1, pool size 210, queued tasks 0

2016-04-29 07:50:28,172 [TaskScheduler-AuthService-DefaultScheduler-004]  INFO  TaskScheduler.java (line 437) Thread stats - total threads in application 248, total tasks 0

[root@ecssinglenode /]#

Nothing regarding my login attempt from just now, may second.

What are the steps required to startup the whole ECS stuff on the single node installation?

Maybe just the startup of these are not done automatically.

Are there any ps | grep commands to find out of the necessary processes are running? Service status commands?

I don't see any docker processes with ecs or docker in their command.

Best regards, Holger

The fact that you're missing the entirety of your /opt directory is concerning - it contains a big chunk of your configuration files, so you're likely to see more issues stem from this. Depending on how much data you've already written and the relative simplicity of teardown/rebuild, it might pay to reinstall the node in this case.

The installation should set docker to start itself and the container on restart (service docker start and docker start ecsstandalone), which is all that should be necessary to bring the container up. You can verify these with service docker status and docker ps -a (adding the -a flag will make the output show non-running containers).

In the container itself, you can use service -s to get all services; there should be a list near the bottom that tells you all of the running services. I believe the full list should be

Services running: authsvc stat georeceiver eventsvc blobsvc dataheadsvc provisionsvc resourcesvc rm cm ssm objcontrolsvc metering storageserver dtquery vnest coordinatorsvc ecsportalsvc transformsvc

Hi Aaron

Here is what I did on that machine:

Installation of CentOS minimal edition

sudo yum update

sudo yum install git tar wget

git clone https://github.com/EMCECS/ECS-CommunityEdition

cd ECS-CommunityEdition/ecs-single-node/

sudo python step1_ecs_singlenode_install.py --disks sdb --hostname ecssinglenode --ethadapter eno16777984

sudo python step2_object_provisioning.py --ECSNodes=192.168.1.27 --Namespace=snns1 --ObjectVArray=snova1 --ObjectVPool=snovp1 --UserName=emccode --DataStoreName=snds1 --VDCName=snvdc1 --MethodName=

Everything fine. I could log in, create buckets, data users and so on.

Restarted the machine: no longer able to log in.

Checking the list of services and their status:

[root@ecssinglenode ~]# service --status-all

netconsole module not loaded

Configured devices:

lo eno16777984

Currently active devices:

lo eno16777984

[root@ecssinglenode ~]# chkconfig --list

Note: This output shows SysV services only and does not include native

      systemd services. SysV configuration data might be overridden by native

      systemd configuration.

      If you want to list systemd services use 'systemctl list-unit-files'.

      To see services enabled on particular target use

      'systemctl list-dependencies [target]'.

netconsole      0:off   1:off   2:off   3:off   4:off   5:off   6:off

network         0:off   1:off   2:on    3:on    4:on    5:on    6:off

[root@ecssinglenode ~]#

Could it be that the minimal edition is the problem? I can try a server with gui version if you think it makes a difference. It is just that in one of the responses I saw that a minimal edition should be fine.

Best regards, Holger

Holger;

Sorry, I should have been more specific: from the VM on which the docker container is running, you can enter the container using the command nsenter -i -m -p -u -t `pidof blobsvc` and from there use the service --status-all command to get the list of services running on the ECS.

A minimal installation of CentOS should be absolutely fine for ECSCE; it's what we mostly use for testing.

- Aaron

Hi Aaron

Thanks for your continued help. But this stuff really s***.

Initial install works, I can connect. But run in to internal errors when creating users and buckets.

Then no reconnect is possible anymore. Rebooting leads to the same issue not being able to connect.

Is there anybody out there who has really gotten this to work? I'm happy to pay up to one day for the guy who gets a virtual ecs single node up and running with a user and bucket configuration. The environment will have to survive reboots of the vm and the installation steps have to be documented and repeatable.

See above for what I've don on my ESX Server. Access via Webex or Teamviewer possible.

In case it helps anybody. Issued a service docker status after the reboot of the VM: Guess what, docker was not running. It really looks like EMC still needs to work quite a bit on these install and other scripts. Reinstalled today with the latest online EMCECS version.

Best regards, Holger

Hi Aaron

What are the steps required to startup ECS after a vm restart?

sudo docker start to begin with as no docker services are running.

Then what?

I tried docker run -d docker.io/emccorp/ecs-software-2.2 but a docker ps showed no running ecs container.

Thanks, Holger

Hi Aaron

Great. After rebooting the machine, I issue:

service docker start

docker start ecsstandalone

Then I can log on to ecs as I did after the initial installation.

Thanks a lot for your assistance. If this would make it into the documentation, that may help others.

Best regards, Holger

Hi Brian

Here is what I did and which now works:

Install CentOS 7 minimal

sudo yum update

sudo yum install git tar wget

git clone https://github.com/EMCECS/ECS-CommunityEdition

cd ECS-CommunityEdition/ecs-single-node/

sudo python step1_ecs_singlenode_install.py --disks sdb --hostname ecssinglenode --ethadapter eno16777984

sudo python step2_object_provisioning.py --ECSNodes=192.168.1.27 --Namespace=snns1 --ObjectVArray=snova1 --ObjectVPool=snovp1 --UserName=emccode --DataStoreName=snds1 --VDCName=snvdc1 --MethodName=

After restarting the machine:

sudo service docker start

sudo docker start ecsstandalone