Wiki » History » Version 17
« Previous -
Version 17/19
(diff) -
Next » -
Current version
Pablo Alingery, 16/10/2017 16:14
Installation Netdrms_9.0¶
Create dir¶
Create the directory to untar netdrms_9.0 from http://jsoc.stanford.edu/~arta/for-pablo/netdrms-9.X.20170530tar.gz
mkdir /usr/local/netdrms_9.0
Create the directory netdmrs_9.1.1 (location Art's home directory at Stanford ) as root
mkdir /usr/local/netdrms_9.0
Change owner for production:ias
chown -R production:ias netdrms_9.1.1
Configuration file¶
cp config.local.template config.local
Config from netdmrs_8.12
Warning : Config not done for remote sum as jmd is used
File configure modif¶
In file /usr/local/netdrms_current/configure
Replace all " $? !=0 " by " $status !=0 " in file configure then tape
File jsoc_machine.csh modif¶
In file /usr/local/netdrms_current/build/jsoc_machine.csh (Same issue in NetDrms 8.11 )
ligne 28 :
case "x86_64": echo linux_x86_64 breaksw
File make_basic.mk¶
In file make_basic.mk line 223
look for "GCC_LF_ALL ="
add "-lcrypto"
Execution configure¶
./configure
Build¶
Type
make make sums
For netdrms_9.1.1 as we are going to use new remote sums (written in python)
we only need to do the following
make show_info make show_series
Checking the modif on Db from NetDrms 2.4 we have figured out that some index where added
The other modif are for sum_rm use ...
Not needed at IAS
On sdo and sdo3 modification in the DB ( index addition)
production@sdo:~$ psql ias_sdo_sums -p5434 ias_sdo_sums=# CREATE INDEX sum_partn_alloc_wd_idx ON public.sum_partn_alloc(wd) ;
Install OK
Starting Netdrms¶
Starting it I have got : production@sdo3:/usr/local/netdrms_current$ sum_start.NetDRMS Number of SUM process sets to spawn = 1 Max number of SUM process can spawn = 8 SUMSERVER is sdo3 SUMS_MANAGER is production SUMS db is ias_sdo_sums SUMS log dir is /usr/local/netdrms_current/logs/SUM sum_start.NetDRMS at 2017.07.12_19:41:38 ** NOTE: SUMS is already running Do you want me to do a sum_stop.NetDRMS followed by a sum_start.NetDRMS (y or n): y SUMS log dir is /usr/local/netdrms_current/logs/SUM sum_stop.NetDRMS at 2017.07.12_19:41:53 NOTE: This will stop sum_svc and related processes. Make sure you want to do this Enter reason for stop (terminate with single '.' or ^D): >> some reason >> . Please identify yourself: >> Pablo Timed-out waiting for sums_procck.py (pid 12640) to exit. ** Now starting SUMS on ias_sdo_sums SUM NetDRMS Start Complete production@sdo3:/usr/local/netdrms_current$ Running on SUMS-server host sdo3. Connection to SUMS DB ias_sdo_sums. Using SUMS-log file sum_svc_2017.07.12.192435.logR. Using SUMS-rm-log file 2017.07.12.192435.log. Process sum_svc is not running. Starting process sum_svc. Please wait for sum_svc and tape inventory (optional) to initialize... Can't open config file /usr/local/netdrms_current/logs/SUM/sum_rm.cfg. Using defaults... rm: impossible de supprimer « /SUM12/D355906038/Records.txt »: Système de fichiers accessible en lecture seulement rm: impossible de supprimer « /SUM12/D355906038/S00000/spikes.fits »: Système de fichiers accessible en lecture seulement rm: impossible de supprimer « /SUM12/D355906038/S00000/image_lev1.fits »: Système de fichiers accessible en lecture seulement (...)
So I killed netdrms to fix that point
Shutting down SUMS... Killing [sum_rm](pid 17376). Killing sum_svc(pid 17371). Removing pidfile /usr/local/netdrms_current/logs/SUM/sums.pidfile Exiting process via shutdown() with return value 0.
The sum_rm is running although in config.local I have set SUMRM_NOOP to 1
# If the value is set to anything other than 0, then sum_rm is rendered inactive. Otherwise, sum_rm is active. SUMRM_NOOP 1
So I put an old sum_rm.cfg in /home/production/netdrms_current/logs/SUM/ then it was ok
production@sdo3:/usr/local/netdrms_current$ sum_start.NetDRMS Number of SUM process sets to spawn = 1 Max number of SUM process can spawn = 8 SUMSERVER is sdo3 SUMS_MANAGER is production SUMS db is ias_sdo_sums SUMS log dir is /usr/local/netdrms_current/logs/SUM sum_start.NetDRMS at 2017.07.12_19:46:51 ** Now starting SUMS on ias_sdo_sums SUM NetDRMS Start Complete production@sdo3:/usr/local/netdrms_current$ Running on SUMS-server host sdo3. Connection to SUMS DB ias_sdo_sums. Using SUMS-log file sum_svc_2017.07.12.194651.log. Using SUMS-rm-log file 2017.07.12.194651.log. Process sum_svc is not running. Starting process sum_svc. Please wait for sum_svc and tape inventory (optional) to initialize... Can't open the log file /home/production/netdrms_current/logs/SUM/sum_rm.log.2017.07.12.194651 for sum_rm on sdo3 ## sum_rm for production pid = 17672 ## You are configured as the active user cmd: cat /tmp/cmdgrep.17672.log | wc -l 1> /tmp/sum_rm.17672.log Connect to Postgres host=sdo3 db=ias_sdo_sums user=production Ok DS_PavailRequest Called alrm_sig() Jul 12 19:46:53 NOOP production@sdo3:/usr/local/netdrms_current$ sum_svc now available
Test Netdrms_9.0¶
Quick test on the thing that bothered us with netdrms_2.4 and lead us to think about an update :
production@sdo3:/usr/local/netdrms_current$ show_info -r -S -P 'hmi.sharp_720s_nrt[5398][! DATE__OBS>$(2016-12-20T23:00:00Z) AND DATE__OBS<$(2016-12-21T01:00:00Z) !]' key='DATE__OBS' recnum sunum DATE__OBS SUDIR 2190405 884413026 2016-12-20T23:10:30.80Z /SUM13/D884413026/S00000 2190409 884414163 2016-12-20T23:22:30.80Z /SUM13/D884414163/S00000 2190413 884416279 2016-12-20T23:34:30.80Z /SUM13/D884416279/S00000 2190417 884417281 2016-12-20T23:46:30.80Z /SUM13/D884417281/S00000 2190421 884419003 2016-12-20T23:58:30.80Z /SUM13/D884419003/S00000 2190425 884421131 2016-12-21T00:10:30.90Z /SUM13/D884421131/S00000 2190429 884422598 2016-12-21T00:22:30.90Z /SUM13/D884422598/S00000 2190433 884424336 2016-12-21T00:34:30.90Z /SUM13/D884424336/S00000 2190437 884425290 2016-12-21T00:46:30.90Z /SUM13/D884425290/S00000 2190441 884427329 2016-12-21T00:58:30.90Z /SUM13/D884427329/S00000
Use sumsd.py , rsumsd.py & sums-clientd.py¶
- sumsd.py is the replacement for sum_svc. It runs on the 'sums server'. This is sdo1.
sumsd.py will open socket to communicate from the client to the server
- rsumsd.py wiil need a couple of table to work so let's create it
This is remote sums; it also runs on sdo1. To use this, both the client machine (sol-calcul4) and the server machine (sdo1) will need access to the DB running on sdo1.
So, for this to be working, you should be able to do a psql -h sdo1 ... from sol-calcul4 and connect.
- sums-clientd.py
will take sunum in the table drms.ingested_sunums to insert them into drms.rs_requests
create missing tables¶
To be done too on sdo1
/usr/local/netdrms_current$ base/drms/scripts/rscreatetabs.py op=create tabs=req,su
ok
ssh-agent¶
ssh-agent was not working so we did regenerate public key
ssh-keygen -t rsa
then copy public key id_rsa.pub into ~/.ssh/authorized_keys on the targeted server jsocport.stanford.edu
jsoc side, on jsocport.stanford.edu
make sure sdo3 ip is referenced in /etc/hosts.netdrms
Launch sumsd.py¶
As 'production'
sumsd.py --loglevel=debug
Launch rsumsd.py¶
The new remote sums server
As 'production'
rsumsd.py --loglevel=debug
create shadows table to be done on sdo1