Project

General

Profile

Wiki » History » Version 11

Pablo Alingery, 27/09/2017 10:29

1 1 Pablo Alingery
h1. Installation Netdrms_9.0
2 1 Pablo Alingery
3 1 Pablo Alingery
4 5 Pablo Alingery
5 1 Pablo Alingery
h2. Create dir 
6 1 Pablo Alingery
7 2 Pablo Alingery
Create the directory to untar netdrms_9.0  from  http://jsoc.stanford.edu/~arta/for-pablo/netdrms-9.X.20170530tar.gz
8 5 Pablo Alingery
9 1 Pablo Alingery
<pre>
10 1 Pablo Alingery
mkdir /usr/local/netdrms_9.0
11 5 Pablo Alingery
</pre>
12 5 Pablo Alingery
13 5 Pablo Alingery
Create the directory netdmrs_9.1.1 (location Art's home directory at Stanford )  as root 
14 5 Pablo Alingery
15 5 Pablo Alingery
<pre>
16 5 Pablo Alingery
mkdir /usr/local/netdrms_9.0
17 5 Pablo Alingery
</pre>
18 5 Pablo Alingery
19 5 Pablo Alingery
Change owner for production:ias 
20 5 Pablo Alingery
21 5 Pablo Alingery
<pre>
22 5 Pablo Alingery
chown -R production:ias netdrms_9.1.1 
23 1 Pablo Alingery
</pre>
24 1 Pablo Alingery
25 1 Pablo Alingery
26 1 Pablo Alingery
h2. Configuration file 
27 1 Pablo Alingery
28 1 Pablo Alingery
<pre>
29 1 Pablo Alingery
cp config.local.template config.local
30 1 Pablo Alingery
</pre>
31 1 Pablo Alingery
32 1 Pablo Alingery
Config from netdmrs_8.12 
33 1 Pablo Alingery
Warning : Config not done for remote sum as jmd is used 
34 1 Pablo Alingery
35 1 Pablo Alingery
36 1 Pablo Alingery
h2. File configure modif
37 1 Pablo Alingery
38 1 Pablo Alingery
In file /usr/local/netdrms_current/configure
39 1 Pablo Alingery
40 1 Pablo Alingery
Replace all " $? !=0 " by " $status !=0 " in file configure then tape
41 1 Pablo Alingery
42 1 Pablo Alingery
43 1 Pablo Alingery
h2. File jsoc_machine.csh modif 
44 1 Pablo Alingery
45 1 Pablo Alingery
In file /usr/local/netdrms_current/build/jsoc_machine.csh (Same issue in NetDrms 8.11 )
46 1 Pablo Alingery
47 1 Pablo Alingery
ligne 28 :
48 1 Pablo Alingery
49 1 Pablo Alingery
<pre>
50 1 Pablo Alingery
case "x86_64":
51 1 Pablo Alingery
      echo linux_x86_64
52 1 Pablo Alingery
      breaksw
53 1 Pablo Alingery
</pre>
54 1 Pablo Alingery
55 1 Pablo Alingery
h2. File make_basic.mk 
56 1 Pablo Alingery
57 1 Pablo Alingery
In file make_basic.mk line 223
58 1 Pablo Alingery
59 1 Pablo Alingery
look for "GCC_LF_ALL ="
60 1 Pablo Alingery
add "-lcrypto"
61 1 Pablo Alingery
62 1 Pablo Alingery
h2. Execution configure
63 4 Pablo Alingery
64 3 Pablo Alingery
<pre>
65 3 Pablo Alingery
./configure
66 3 Pablo Alingery
</pre>
67 1 Pablo Alingery
68 4 Pablo Alingery
h2. Build 
69 4 Pablo Alingery
70 3 Pablo Alingery
Type 
71 1 Pablo Alingery
72 3 Pablo Alingery
<pre>
73 3 Pablo Alingery
make
74 3 Pablo Alingery
make sums
75 3 Pablo Alingery
</pre>
76 1 Pablo Alingery
77 6 Pablo Alingery
78 6 Pablo Alingery
79 11 Pablo Alingery
For netdrms9.1.1 as we are going to use new remote sums (written in python) 
80 11 Pablo Alingery
we only need to do the following
81 6 Pablo Alingery
82 6 Pablo Alingery
<pre>
83 6 Pablo Alingery
make show_info 
84 6 Pablo Alingery
make show_series 
85 6 Pablo Alingery
</pre>
86 6 Pablo Alingery
87 6 Pablo Alingery
Checking the modif on Db from NetDrms 2.4 we have figured out that some index where added
88 6 Pablo Alingery
The other modif are for sum_rm use ...
89 6 Pablo Alingery
Not needed at IAS  
90 6 Pablo Alingery
91 6 Pablo Alingery
On sdo modification of DB  add index 
92 6 Pablo Alingery
93 6 Pablo Alingery
<pre>
94 6 Pablo Alingery
production@sdo:~$ psql ias_sdo_sums -p5434 
95 6 Pablo Alingery
ias_sdo_sums=# CREATE INDEX sum_partn_alloc_wd_idx ON public.sum_partn_alloc(wd) ;
96 6 Pablo Alingery
</pre>
97 6 Pablo Alingery
98 1 Pablo Alingery
99 6 Pablo Alingery
100 11 Pablo Alingery
Install OK
101 1 Pablo Alingery
102 1 Pablo Alingery
h2. Starting  Netdrms 
103 1 Pablo Alingery
104 1 Pablo Alingery
<pre>
105 1 Pablo Alingery
Starting it I have got :
106 1 Pablo Alingery
production@sdo3:/usr/local/netdrms_current$ sum_start.NetDRMS
107 1 Pablo Alingery
Number of SUM process sets to spawn = 1
108 1 Pablo Alingery
Max number of SUM process can spawn = 8
109 1 Pablo Alingery
SUMSERVER is sdo3
110 1 Pablo Alingery
SUMS_MANAGER is production
111 1 Pablo Alingery
SUMS db is ias_sdo_sums
112 1 Pablo Alingery
SUMS log dir is /usr/local/netdrms_current/logs/SUM
113 1 Pablo Alingery
sum_start.NetDRMS at 2017.07.12_19:41:38
114 1 Pablo Alingery
115 1 Pablo Alingery
** NOTE: SUMS is already running
116 1 Pablo Alingery
Do you want me to do a sum_stop.NetDRMS followed by a sum_start.NetDRMS (y or n):
117 1 Pablo Alingery
y
118 1 Pablo Alingery
SUMS log dir is /usr/local/netdrms_current/logs/SUM
119 1 Pablo Alingery
120 1 Pablo Alingery
sum_stop.NetDRMS at 2017.07.12_19:41:53
121 1 Pablo Alingery
122 1 Pablo Alingery
NOTE: This will stop sum_svc and related processes. Make sure you want to do this
123 1 Pablo Alingery
Enter reason for stop (terminate with single '.' or ^D):
124 1 Pablo Alingery
>> some reason
125 1 Pablo Alingery
>> .
126 1 Pablo Alingery
Please identify yourself:
127 1 Pablo Alingery
>> Pablo
128 1 Pablo Alingery
Timed-out waiting for sums_procck.py (pid 12640) to exit.
129 1 Pablo Alingery
** Now starting SUMS on ias_sdo_sums
130 1 Pablo Alingery
SUM NetDRMS Start Complete
131 1 Pablo Alingery
production@sdo3:/usr/local/netdrms_current$ Running on SUMS-server host sdo3.
132 1 Pablo Alingery
Connection to SUMS DB ias_sdo_sums.
133 1 Pablo Alingery
Using SUMS-log file sum_svc_2017.07.12.192435.logR.
134 1 Pablo Alingery
Using SUMS-rm-log file 2017.07.12.192435.log.
135 1 Pablo Alingery
Process sum_svc is not running.
136 1 Pablo Alingery
Starting process sum_svc.
137 1 Pablo Alingery
138 1 Pablo Alingery
Please wait for sum_svc and tape inventory (optional) to initialize...
139 1 Pablo Alingery
Can't open config file /usr/local/netdrms_current/logs/SUM/sum_rm.cfg. Using defaults...
140 1 Pablo Alingery
rm: impossible de supprimer « /SUM12/D355906038/Records.txt »: Système de fichiers accessible en lecture seulement
141 1 Pablo Alingery
rm: impossible de supprimer « /SUM12/D355906038/S00000/spikes.fits »: Système de fichiers accessible en lecture seulement
142 1 Pablo Alingery
rm: impossible de supprimer « /SUM12/D355906038/S00000/image_lev1.fits »: Système de fichiers accessible en lecture seulement
143 1 Pablo Alingery
(...)
144 1 Pablo Alingery
</pre>
145 1 Pablo Alingery
146 1 Pablo Alingery
So I killed netdrms to fix that point 
147 1 Pablo Alingery
148 1 Pablo Alingery
<pre>
149 1 Pablo Alingery
Shutting down SUMS...
150 1 Pablo Alingery
Killing [sum_rm](pid 17376).
151 1 Pablo Alingery
Killing sum_svc(pid 17371).
152 1 Pablo Alingery
Removing pidfile /usr/local/netdrms_current/logs/SUM/sums.pidfile
153 1 Pablo Alingery
Exiting process via shutdown() with return value 0.
154 1 Pablo Alingery
</pre>
155 1 Pablo Alingery
156 1 Pablo Alingery
157 1 Pablo Alingery
The sum_rm is running although in config.local I have set SUMRM_NOOP to 1
158 1 Pablo Alingery
159 1 Pablo Alingery
<pre>
160 1 Pablo Alingery
# If the value is set to anything other than 0, then sum_rm is rendered inactive. Otherwise, sum_rm is active.
161 1 Pablo Alingery
SUMRM_NOOP                             1
162 1 Pablo Alingery
</pre>
163 1 Pablo Alingery
164 1 Pablo Alingery
So I put an old sum_rm.cfg in /home/production/netdrms_current/logs/SUM/ then it was ok
165 1 Pablo Alingery
166 1 Pablo Alingery
<pre>
167 1 Pablo Alingery
production@sdo3:/usr/local/netdrms_current$ sum_start.NetDRMS
168 1 Pablo Alingery
Number of SUM process sets to spawn = 1
169 1 Pablo Alingery
Max number of SUM process can spawn = 8
170 1 Pablo Alingery
SUMSERVER is sdo3
171 1 Pablo Alingery
SUMS_MANAGER is production
172 1 Pablo Alingery
SUMS db is ias_sdo_sums
173 1 Pablo Alingery
SUMS log dir is /usr/local/netdrms_current/logs/SUM
174 1 Pablo Alingery
sum_start.NetDRMS at 2017.07.12_19:46:51
175 1 Pablo Alingery
176 1 Pablo Alingery
** Now starting SUMS on ias_sdo_sums
177 1 Pablo Alingery
SUM NetDRMS Start Complete
178 1 Pablo Alingery
production@sdo3:/usr/local/netdrms_current$ Running on SUMS-server host sdo3.
179 1 Pablo Alingery
Connection to SUMS DB ias_sdo_sums.
180 1 Pablo Alingery
Using SUMS-log file sum_svc_2017.07.12.194651.log.
181 1 Pablo Alingery
Using SUMS-rm-log file 2017.07.12.194651.log.
182 1 Pablo Alingery
Process sum_svc is not running.
183 1 Pablo Alingery
Starting process sum_svc.
184 1 Pablo Alingery
185 1 Pablo Alingery
Please wait for sum_svc and tape inventory (optional) to initialize...
186 1 Pablo Alingery
Can't open the log file /home/production/netdrms_current/logs/SUM/sum_rm.log.2017.07.12.194651 for sum_rm on sdo3
187 1 Pablo Alingery
188 1 Pablo Alingery
189 1 Pablo Alingery
## sum_rm for production  pid = 17672 ##
190 1 Pablo Alingery
You are configured as the active user
191 1 Pablo Alingery
cmd: cat /tmp/cmdgrep.17672.log | wc -l 1> /tmp/sum_rm.17672.log
192 1 Pablo Alingery
Connect to Postgres host=sdo3 db=ias_sdo_sums user=production Ok
193 1 Pablo Alingery
DS_PavailRequest
194 1 Pablo Alingery
Called alrm_sig()
195 1 Pablo Alingery
Jul 12 19:46:53
196 1 Pablo Alingery
NOOP
197 1 Pablo Alingery
198 1 Pablo Alingery
production@sdo3:/usr/local/netdrms_current$ sum_svc now available
199 1 Pablo Alingery
</pre>
200 1 Pablo Alingery
201 1 Pablo Alingery
h2. Test Netdrms_9.0
202 1 Pablo Alingery
203 1 Pablo Alingery
Quick test on the thing that bothered us with netdrms_2.4 and lead us to think about an update :
204 1 Pablo Alingery
205 1 Pablo Alingery
<pre>
206 1 Pablo Alingery
production@sdo3:/usr/local/netdrms_current$ show_info -r -S -P 'hmi.sharp_720s_nrt[5398][! DATE__OBS>$(2016-12-20T23:00:00Z) AND DATE__OBS<$(2016-12-21T01:00:00Z) !]' key='DATE__OBS'
207 1 Pablo Alingery
recnum    sunum    DATE__OBS    SUDIR
208 1 Pablo Alingery
2190405    884413026    2016-12-20T23:10:30.80Z    /SUM13/D884413026/S00000
209 1 Pablo Alingery
2190409    884414163    2016-12-20T23:22:30.80Z    /SUM13/D884414163/S00000
210 1 Pablo Alingery
2190413    884416279    2016-12-20T23:34:30.80Z    /SUM13/D884416279/S00000
211 1 Pablo Alingery
2190417    884417281    2016-12-20T23:46:30.80Z    /SUM13/D884417281/S00000
212 1 Pablo Alingery
2190421    884419003    2016-12-20T23:58:30.80Z    /SUM13/D884419003/S00000
213 1 Pablo Alingery
2190425    884421131    2016-12-21T00:10:30.90Z    /SUM13/D884421131/S00000
214 1 Pablo Alingery
2190429    884422598    2016-12-21T00:22:30.90Z    /SUM13/D884422598/S00000
215 1 Pablo Alingery
2190433    884424336    2016-12-21T00:34:30.90Z    /SUM13/D884424336/S00000
216 1 Pablo Alingery
2190437    884425290    2016-12-21T00:46:30.90Z    /SUM13/D884425290/S00000
217 1 Pablo Alingery
2190441    884427329    2016-12-21T00:58:30.90Z    /SUM13/D884427329/S00000
218 1 Pablo Alingery
</pre>
219 7 Pablo Alingery
220 7 Pablo Alingery
h1. Use sumsd.py
221 7 Pablo Alingery
222 7 Pablo Alingery
sumsd.py will open socket to communicate from the client to the server 
223 7 Pablo Alingery
rsumsd.py wiil need a couple of table to work  so let's create it 
224 1 Pablo Alingery
225 1 Pablo Alingery
h2. create missing tables 
226 9 Pablo Alingery
227 9 Pablo Alingery
To be done too on sdo1 
228 9 Pablo Alingery
229 1 Pablo Alingery
<pre>
230 7 Pablo Alingery
/usr/local/netdrms_current$ base/drms/scripts/rscreatetabs.py op=create  tabs=req,su
231 7 Pablo Alingery
</pre>
232 9 Pablo Alingery
233 10 Pablo Alingery
ok
234 10 Pablo Alingery
235 8 Pablo Alingery
h2. ssh-agent
236 9 Pablo Alingery
237 9 Pablo Alingery
ssh-agent was not working so we did regenerate public key 
238 9 Pablo Alingery
239 1 Pablo Alingery
<pre>
240 8 Pablo Alingery
ssh-keygen -t rsa
241 8 Pablo Alingery
</pre>
242 8 Pablo Alingery
243 1 Pablo Alingery
244 8 Pablo Alingery
then copy public key id_rsa.pub into ~/.ssh/authorized_keys on the targeted server  jsocport.stanford.edu
245 8 Pablo Alingery
246 10 Pablo Alingery
jsoc side, on  jsocport.stanford.edu
247 10 Pablo Alingery
make sure sdo3 ip is referenced in /etc/hosts.netdrms