NetDRMS Installation » History » Version 368
Pablo Alingery, 13/09/2018 15:43
1 | 50 | Herve Ballans | {{toc}} |
---|---|---|---|
2 | 1 | Herve Ballans | |
3 | 351 | Pablo Alingery | h1. NetDRMS 8.12 Installation |
4 | 1 | Herve Ballans | |
5 | 276 | Herve Ballans | h2. Preliminary note |
6 | 275 | Herve Ballans | |
7 | 275 | Herve Ballans | In the following sections, when there is a command line : |
8 | 324 | Herve Ballans | <pre>#</pre>means that command is executed into _root_ user |
9 | 324 | Herve Ballans | <pre>$</pre>means that command is executed into _production_ user |
10 | 275 | Herve Ballans | |
11 | 333 | Herve Ballans | h2. Environment |
12 | 333 | Herve Ballans | |
13 | 343 | Herve Ballans | Server : DELL PowerEdge R730 |
14 | 343 | Herve Ballans | RAM : 64.00GB |
15 | 343 | Herve Ballans | CPU : 32 CPUs Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz |
16 | 345 | Herve Ballans | |
17 | 345 | Herve Ballans | Storage Configuration : |
18 | 344 | Herve Ballans | * 264 GB mounted in / (2 X 278.88 GB SAS 12G disks in RAID1) |
19 | 344 | Herve Ballans | * 1,7 TB mounted in /var/lib/postgresql (7 X 278.88 GB SAS 12G disks in RAID5 configuration) |
20 | 344 | Herve Ballans | * 1 global spare for both RAID |
21 | 344 | Herve Ballans | * DDP storage mounted in /SUM* |
22 | 343 | Herve Ballans | |
23 | 343 | Herve Ballans | Guest OS : Debian 8.4 |
24 | 343 | Herve Ballans | Kernel : Linux sdo-new 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-1 (2016-03-06) x86_64 GNU/Linux |
25 | 343 | Herve Ballans | |
26 | 1 | Herve Ballans | h2. Requirements |
27 | 1 | Herve Ballans | |
28 | 47 | Pablo Alingery | h3. Configuring production environment |
29 | 1 | Herve Ballans | |
30 | 284 | Herve Ballans | Adding user production |
31 | 47 | Pablo Alingery | |
32 | 1 | Herve Ballans | <pre> |
33 | 1 | Herve Ballans | # adduser production |
34 | 1 | Herve Ballans | # addgroup ias |
35 | 1 | Herve Ballans | # adduser production ias |
36 | 1 | Herve Ballans | </pre> |
37 | 1 | Herve Ballans | |
38 | 49 | Herve Ballans | h3. Configure /home/production/.profile |
39 | 1 | Herve Ballans | |
40 | 49 | Herve Ballans | Adding environment variables |
41 | 1 | Herve Ballans | |
42 | 1 | Herve Ballans | <pre> |
43 | 122 | Herve Ballans | #Modif Herve.B Pablo.A 12/05/2016 |
44 | 122 | Herve Ballans | |
45 | 1 | Herve Ballans | PATH="/usr/local/netdrms_current/bin/linux_x86_64:$PATH" |
46 | 49 | Herve Ballans | PATH="/usr/local/netdrms_current/scripts:$PATH" |
47 | 122 | Herve Ballans | PATH="/usr/local/jmd/bin:$PATH" |
48 | 122 | Herve Ballans | PATH="/usr/local/jmd/scripts:$PATH" |
49 | 141 | Pablo Alingery | #The following is not needed as installed with 'apt-get install' |
50 | 141 | Pablo Alingery | #PATH="/usr/local/pgsql/bin:$PATH" |
51 | 122 | Herve Ballans | PATH="/usr/local/netdrms-tools/scripts:$PATH" |
52 | 49 | Herve Ballans | #2014-12-17 Avec Herve et Pablo on pense que la ligne suivante est intutile as it is f.. install in /usr/local/lib |
53 | 140 | Pablo Alingery | PATH="/usr/local/cfitsio:$PATH" |
54 | 1 | Herve Ballans | #Done |
55 | 49 | Herve Ballans | |
56 | 122 | Herve Ballans | DERBY_HOME=/usr/local/jmd/databases/derby/derbyBD |
57 | 49 | Herve Ballans | export DERBY_HOME |
58 | 49 | Herve Ballans | |
59 | 49 | Herve Ballans | export CVSROOT=:pserver:anonymous@solarch.tuc.noao.edu:2401/vtarc1/vso/cvsroot |
60 | 49 | Herve Ballans | </pre> |
61 | 49 | Herve Ballans | |
62 | 162 | Pablo Alingery | h3. Configure /varl/lib/postgres/.profile |
63 | 161 | Pablo Alingery | |
64 | 163 | Pablo Alingery | Add to $PATH so postgres can exeute command like initdb |
65 | 161 | Pablo Alingery | |
66 | 161 | Pablo Alingery | <pre> |
67 | 161 | Pablo Alingery | #Modif Herve.B Pablo.A 19/05/2016 |
68 | 161 | Pablo Alingery | PATH="/usr/lib/postgresql/9.4/bin:$PATH" |
69 | 161 | Pablo Alingery | </pre> |
70 | 161 | Pablo Alingery | |
71 | 125 | Pablo Alingery | h3. Setting privileges for SUMS_MANAGER |
72 | 87 | Herve Ballans | |
73 | 365 | Pablo Alingery | Please modify file /etc/sudoers so user 'production' will be able to execute 'sum_chown' that we will located in : /usr/local/bin . See below |
74 | 365 | Pablo Alingery | |
75 | 125 | Pablo Alingery | Add to /etc/sudoers the following line : |
76 | 1 | Herve Ballans | production host=NOPASSWD:/usr/local/bin/sum_chmown |
77 | 1 | Herve Ballans | |
78 | 127 | Pablo Alingery | h3. Set passwords file for db postgres |
79 | 1 | Herve Ballans | |
80 | 127 | Pablo Alingery | Please create file .pgpass with the following information for user 'production" so he will be able to connect without password as 'production' or 'postgres' to 2 db that we will create later ias_sdo and ias_sdo_sums. |
81 | 1 | Herve Ballans | |
82 | 1 | Herve Ballans | |
83 | 1 | Herve Ballans | .pgpass content : |
84 | 1 | Herve Ballans | <pre> |
85 | 128 | Pablo Alingery | #hostname:5432:ias_sdo:production: |
86 | 128 | Pablo Alingery | #hostname:5434:ias_sdo_sums:production: |
87 | 366 | Pablo Alingery | #hostname:5436:ias_sdo_monitor:production: |
88 | 128 | Pablo Alingery | #hostname:5432:ias_sdo:postgres: |
89 | 128 | Pablo Alingery | #hostname:5434:ias_sdo_sums:postgres |
90 | 366 | Pablo Alingery | #hostname:5436:ias_sdo_monitor:postgres: |
91 | 1 | Herve Ballans | </pre> |
92 | 128 | Pablo Alingery | |
93 | 129 | Pablo Alingery | where #hostname is the name of your server |
94 | 129 | Pablo Alingery | |
95 | 129 | Pablo Alingery | |
96 | 131 | Pablo Alingery | %{color:red}WARNING : change hostname when migrate sdo-new to sdo% |
97 | 1 | Herve Ballans | |
98 | 133 | Pablo Alingery | h3. Install missing libraries |
99 | 1 | Herve Ballans | |
100 | 293 | Herve Ballans | * libreadline-dev (but already installed with postgresql) |
101 | 293 | Herve Ballans | * libssl-dev |
102 | 293 | Herve Ballans | * libpam0g-dev |
103 | 293 | Herve Ballans | * libperl-dev |
104 | 293 | Herve Ballans | * libnet-ssh-perl |
105 | 293 | Herve Ballans | * libstring-shellquote-perl |
106 | 296 | Herve Ballans | * libdbi-perl |
107 | 296 | Herve Ballans | * libdbd-pg-perl |
108 | 1 | Herve Ballans | |
109 | 133 | Pablo Alingery | NB : Already done in Debian 8.4 Jessy |
110 | 1 | Herve Ballans | <pre> |
111 | 1 | Herve Ballans | # apt-get install $library_name |
112 | 1 | Herve Ballans | </pre> |
113 | 1 | Herve Ballans | where |
114 | 1 | Herve Ballans | _$library_name_ is the name of library |
115 | 19 | Pablo Alingery | |
116 | 1 | Herve Ballans | Then to update path on libraries, please type : |
117 | 1 | Herve Ballans | <pre> |
118 | 19 | Pablo Alingery | # ldconfig |
119 | 1 | Herve Ballans | </pre> |
120 | 1 | Herve Ballans | |
121 | 1 | Herve Ballans | h3. Reconfigure locale |
122 | 1 | Herve Ballans | |
123 | 1 | Herve Ballans | <pre> |
124 | 1 | Herve Ballans | # dpkg-reconfigure locales |
125 | 1 | Herve Ballans | </pre> |
126 | 1 | Herve Ballans | |
127 | 1 | Herve Ballans | select fr_FR.UTF8 UTF8 and unselect en_US.UTF-8 |
128 | 1 | Herve Ballans | |
129 | 1 | Herve Ballans | localedef -i en_US -f UTF-8 en_US.UTF-8 |
130 | 1 | Herve Ballans | |
131 | 1 | Herve Ballans | (see here : [[http://www.thomas-krenn.com/en/wiki/Perl_warning_Setting_locale_failed_in_Debian]]) |
132 | 1 | Herve Ballans | |
133 | 134 | Pablo Alingery | h3. Install gfortran packages |
134 | 1 | Herve Ballans | |
135 | 1 | Herve Ballans | <pre> |
136 | 1 | Herve Ballans | # apt-get install gfortran swig |
137 | 1 | Herve Ballans | </pre> |
138 | 1 | Herve Ballans | |
139 | 1 | Herve Ballans | h3. Installation of cfitsio library |
140 | 1 | Herve Ballans | |
141 | 135 | Pablo Alingery | In /usr/local dir |
142 | 135 | Pablo Alingery | |
143 | 135 | Pablo Alingery | Version 3.39 (currently on sdo : 3.24) |
144 | 1 | Herve Ballans | <pre> |
145 | 135 | Pablo Alingery | # wget ftp://heasarc.gsfc.nasa.gov/software/fitsio/c/cfitsio3390.tar.gz |
146 | 1 | Herve Ballans | </pre> |
147 | 137 | Pablo Alingery | Untar and decompression |
148 | 1 | Herve Ballans | <pre> |
149 | 136 | Pablo Alingery | # tar -xzvf cfitsio3390.tar.gz |
150 | 1 | Herve Ballans | </pre> |
151 | 1 | Herve Ballans | |
152 | 139 | Pablo Alingery | Go into /usr/local and type |
153 | 1 | Herve Ballans | |
154 | 1 | Herve Ballans | su - root and : |
155 | 1 | Herve Ballans | <pre> |
156 | 1 | Herve Ballans | # ./configure --prefix=/usr/local |
157 | 1 | Herve Ballans | # make |
158 | 1 | Herve Ballans | # make install |
159 | 135 | Pablo Alingery | </pre> |
160 | 118 | Pablo Alingery | |
161 | 237 | Herve Ballans | h3. Installation des libraries libtar-dev, libcurl3-dev et libecpg-dev |
162 | 237 | Herve Ballans | |
163 | 237 | Herve Ballans | <pre> |
164 | 237 | Herve Ballans | apt-get install libtar libtar-dev |
165 | 237 | Herve Ballans | </pre> |
166 | 237 | Herve Ballans | |
167 | 238 | Pablo Alingery | Note, selecting 'libcurl4-openssl-dev' instead of 'libcurl3-dev' |
168 | 238 | Pablo Alingery | libcurl4-openssl-dev is already the newest version |
169 | 237 | Herve Ballans | <pre> |
170 | 237 | Herve Ballans | apt-get install libcurl3-dev |
171 | 237 | Herve Ballans | </pre> |
172 | 237 | Herve Ballans | |
173 | 237 | Herve Ballans | <pre> |
174 | 237 | Herve Ballans | apt-get install libecpg-dev |
175 | 237 | Herve Ballans | </pre> |
176 | 237 | Herve Ballans | |
177 | 144 | Pablo Alingery | h3. Installation of apache2 and lib perl |
178 | 118 | Pablo Alingery | |
179 | 118 | Pablo Alingery | <pre> |
180 | 118 | Pablo Alingery | # apt-get install apache2 |
181 | 118 | Pablo Alingery | |
182 | 118 | Pablo Alingery | # apt-get install libjson-perl |
183 | 118 | Pablo Alingery | |
184 | 118 | Pablo Alingery | # apt-get install libapache2-mod-perl2 |
185 | 118 | Pablo Alingery | </pre> |
186 | 1 | Herve Ballans | |
187 | 183 | Herve Ballans | h3. Installation of postgresql |
188 | 1 | Herve Ballans | |
189 | 145 | Pablo Alingery | Into 'root' |
190 | 145 | Pablo Alingery | |
191 | 145 | Pablo Alingery | <pre> |
192 | 147 | Pablo Alingery | apt-get install postgresql-9.4 postgresql-client-9.4 |
193 | 145 | Pablo Alingery | </pre> |
194 | 145 | Pablo Alingery | |
195 | 213 | Herve Ballans | h3. Installation of python3 |
196 | 209 | Pablo Alingery | |
197 | 209 | Pablo Alingery | Into 'root' |
198 | 209 | Pablo Alingery | |
199 | 209 | Pablo Alingery | <pre> |
200 | 209 | Pablo Alingery | apt-get install python3 |
201 | 1 | Herve Ballans | </pre> |
202 | 210 | Pablo Alingery | |
203 | 211 | Pablo Alingery | Add some modules psycopg2 (postgres connector) and pySmartDL (Download manager) |
204 | 210 | Pablo Alingery | |
205 | 210 | Pablo Alingery | <pre> |
206 | 210 | Pablo Alingery | apt-get install python3-psycopg2 |
207 | 210 | Pablo Alingery | </pre> |
208 | 210 | Pablo Alingery | |
209 | 210 | Pablo Alingery | Add python3-pip for modules within python3 |
210 | 210 | Pablo Alingery | |
211 | 210 | Pablo Alingery | <pre> |
212 | 210 | Pablo Alingery | apt-get install python3-pip |
213 | 1 | Herve Ballans | </pre> |
214 | 212 | Pablo Alingery | |
215 | 212 | Pablo Alingery | <pre> |
216 | 212 | Pablo Alingery | pip3 install pySmartDL |
217 | 212 | Pablo Alingery | </pre> |
218 | 1 | Herve Ballans | |
219 | 213 | Herve Ballans | Both python2.7 and 3.4 versions are installed on the system. |
220 | 213 | Herve Ballans | By default, the python2.7 is used. |
221 | 367 | Pablo Alingery | |
222 | 367 | Pablo Alingery | Update alternatives table |
223 | 367 | Pablo Alingery | |
224 | 367 | Pablo Alingery | <pre> |
225 | 367 | Pablo Alingery | update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1 |
226 | 367 | Pablo Alingery | update-alternatives --install /usr/bin/python python /usr/bin/python3.4 2 |
227 | 367 | Pablo Alingery | </pre> |
228 | 367 | Pablo Alingery | |
229 | 367 | Pablo Alingery | In order to change in 3.4 by default, type and choose 3.4 typying : |
230 | 213 | Herve Ballans | <pre> |
231 | 213 | Herve Ballans | update-alternatives --config python |
232 | 213 | Herve Ballans | </pre> |
233 | 209 | Pablo Alingery | |
234 | 1 | Herve Ballans | h3. Installation of sdo_scripts |
235 | 218 | Pablo Alingery | |
236 | 368 | Pablo Alingery | Install monitoring scripts for sdo into 'production' |
237 | 218 | Pablo Alingery | |
238 | 1 | Herve Ballans | <pre> |
239 | 368 | Pablo Alingery | sdo_scripts |
240 | 368 | Pablo Alingery | cd .. |
241 | 368 | Pablo Alingery | production@sdo2:~/netdrms-tools$ git clone https://git.ias.u-psud.fr/palinger/sdoscripts.git -b prod-sdo1 |
242 | 218 | Pablo Alingery | </pre> |
243 | 218 | Pablo Alingery | |
244 | 349 | Herve Ballans | h3. lib64 symbolic link |
245 | 349 | Herve Ballans | |
246 | 349 | Herve Ballans | Creating lib64 link (lib64 does not exist on debian 8) |
247 | 349 | Herve Ballans | <pre> |
248 | 349 | Herve Ballans | # cd /usr |
249 | 349 | Herve Ballans | # ln -s lib/ lib64 |
250 | 349 | Herve Ballans | </pre> |
251 | 349 | Herve Ballans | |
252 | 349 | Herve Ballans | |
253 | 1 | Herve Ballans | h2. NetDRMS databases |
254 | 1 | Herve Ballans | |
255 | 156 | Pablo Alingery | into /var/lib/postgresql (hard mount point separated, 7 disks 15000 tr/min in raid5 in order to distribute read and write) |
256 | 1 | Herve Ballans | |
257 | 1 | Herve Ballans | <pre> |
258 | 158 | Pablo Alingery | # cd /var/lib/postgresql |
259 | 157 | Pablo Alingery | # mkdir data data_sums data_monitor |
260 | 157 | Pablo Alingery | # chown -R postgres:postgres * |
261 | 1 | Herve Ballans | </pre> |
262 | 1 | Herve Ballans | |
263 | 160 | Pablo Alingery | h3. Initialize 3 servers data, data_sums, data_monitor |
264 | 1 | Herve Ballans | |
265 | 159 | Pablo Alingery | Into postgres |
266 | 1 | Herve Ballans | <pre> |
267 | 1 | Herve Ballans | # su - postgres |
268 | 1 | Herve Ballans | |
269 | 1 | Herve Ballans | $ initdb --locale=C -D data -A md5 -W |
270 | 1 | Herve Ballans | (no password) |
271 | 2 | Herve Ballans | |
272 | 2 | Herve Ballans | Success. You can now start the database server using: |
273 | 2 | Herve Ballans | |
274 | 2 | Herve Ballans | postgres -D data |
275 | 2 | Herve Ballans | or |
276 | 2 | Herve Ballans | pg_ctl -D data -l logfile start |
277 | 2 | Herve Ballans | |
278 | 1 | Herve Ballans | |
279 | 1 | Herve Ballans | $ initdb --locale=C -D data_sums -A md5 -W |
280 | 2 | Herve Ballans | (no password) |
281 | 2 | Herve Ballans | |
282 | 2 | Herve Ballans | Success. You can now start the database server using: |
283 | 2 | Herve Ballans | |
284 | 2 | Herve Ballans | postgres -D data_sums |
285 | 2 | Herve Ballans | or |
286 | 2 | Herve Ballans | pg_ctl -D data_sums -l logfile start |
287 | 8 | Pablo Alingery | |
288 | 8 | Pablo Alingery | $ initdb --locale=C -D data_monitor -A md5 -W |
289 | 8 | Pablo Alingery | (no password) |
290 | 8 | Pablo Alingery | |
291 | 8 | Pablo Alingery | Success. You can now start the database server using: |
292 | 8 | Pablo Alingery | |
293 | 8 | Pablo Alingery | postgres -D data_monitor |
294 | 8 | Pablo Alingery | or |
295 | 8 | Pablo Alingery | pg_ctl -D data_monitor -l logfile start |
296 | 2 | Herve Ballans | </pre> |
297 | 2 | Herve Ballans | |
298 | 1 | Herve Ballans | |
299 | 1 | Herve Ballans | |
300 | 1 | Herve Ballans | Edition des fichiers postgresql.conf pour configurer les bons ports d'écoute : |
301 | 1 | Herve Ballans | listen_addresses = '*' |
302 | 1 | Herve Ballans | data -> port 5432 |
303 | 10 | Pablo Alingery | data_sums -> port 5434 |
304 | 1 | Herve Ballans | data_monitor -> port 5436 |
305 | 10 | Pablo Alingery | |
306 | 165 | Pablo Alingery | Edit files data*/pg_hba.conf : replace 'md5' by 'trust' |
307 | 1 | Herve Ballans | <pre> |
308 | 1 | Herve Ballans | #"local" |
309 | 1 | Herve Ballans | local all all trust |
310 | 1 | Herve Ballans | #"IPv4" |
311 | 1 | Herve Ballans | host all all trust |
312 | 1 | Herve Ballans | </pre> |
313 | 166 | Pablo Alingery | |
314 | 166 | Pablo Alingery | h3. Tunning postgres |
315 | 166 | Pablo Alingery | |
316 | 166 | Pablo Alingery | In file postgres.conf ( according to https://www.qwant.com/?q=Tunning+postgres+server+9.4+&client=firefox ) |
317 | 166 | Pablo Alingery | |
318 | 168 | Pablo Alingery | shared_buffer=4096MB %{color:red}So 1/12 of the total RAM memory available% |
319 | 1 | Herve Ballans | |
320 | 169 | Pablo Alingery | work_mem = 24MB |
321 | 168 | Pablo Alingery | |
322 | 168 | Pablo Alingery | maintenance_work_mem = 4096MB |
323 | 1 | Herve Ballans | |
324 | 169 | Pablo Alingery | wal_buffers = 16MB |
325 | 169 | Pablo Alingery | |
326 | 169 | Pablo Alingery | checkpoint_segments = 32 |
327 | 169 | Pablo Alingery | |
328 | 169 | Pablo Alingery | checkpoint_completion_target = 0.9 |
329 | 169 | Pablo Alingery | |
330 | 169 | Pablo Alingery | random_page_cost = 1.0 |
331 | 169 | Pablo Alingery | |
332 | 169 | Pablo Alingery | effective_cache_size = 16384MB so 1/4 of the max memory |
333 | 169 | Pablo Alingery | |
334 | 170 | Pablo Alingery | |
335 | 170 | Pablo Alingery | |
336 | 170 | Pablo Alingery | #log |
337 | 169 | Pablo Alingery | logging_collector = on |
338 | 169 | Pablo Alingery | log_directory = 'pg_log' |
339 | 169 | Pablo Alingery | log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # log file name pattern, |
340 | 169 | Pablo Alingery | log_truncate_on_rotation = on # If on, an existing log file with the |
341 | 169 | Pablo Alingery | log_rotation_age = 1d # Automatic rotation of logfiles will |
342 | 170 | Pablo Alingery | client_min_messages = error |
343 | 170 | Pablo Alingery | log_min_messages = error |
344 | 170 | Pablo Alingery | log_min_duration_statement = 0 |
345 | 170 | Pablo Alingery | |
346 | 170 | Pablo Alingery | log_connections = on |
347 | 170 | Pablo Alingery | log_disconnections = on |
348 | 170 | Pablo Alingery | log_duration = off |
349 | 170 | Pablo Alingery | log_hostname = on |
350 | 170 | Pablo Alingery | log_line_prefix = '%t [%p]: %u@%h - %d :' |
351 | 170 | Pablo Alingery | |
352 | 170 | Pablo Alingery | track_counts = on |
353 | 170 | Pablo Alingery | autovacuum = on |
354 | 170 | Pablo Alingery | |
355 | 170 | Pablo Alingery | extra_float_digits = 3 |
356 | 170 | Pablo Alingery | |
357 | 177 | Pablo Alingery | h3. Start the 3 databases : |
358 | 1 | Herve Ballans | |
359 | 1 | Herve Ballans | <pre> |
360 | 1 | Herve Ballans | # su - postgres |
361 | 196 | Herve Ballans | $ cd /var/lib/postgresql |
362 | 172 | Pablo Alingery | $ pg_ctl -D data start |
363 | 172 | Pablo Alingery | $ pg_ctl -D data_sums start |
364 | 172 | Pablo Alingery | $ pg_ctl -D data_monitor start |
365 | 1 | Herve Ballans | </pre> |
366 | 174 | Pablo Alingery | |
367 | 1 | Herve Ballans | h3. Create the 3 databases : |
368 | 181 | Pablo Alingery | |
369 | 178 | Pablo Alingery | Into user 'postgres' |
370 | 1 | Herve Ballans | |
371 | 179 | Pablo Alingery | * createdb --locale C -E LATIN1 -T template0 ias_sdo |
372 | 179 | Pablo Alingery | * createdb --locale C -E LATIN1 -T template0 -p 5434 ias_sdo_sums |
373 | 179 | Pablo Alingery | * createdb --locale C -E LATIN1 -T template0 -p 5436 ias_sdo_monitor |
374 | 54 | Herve Ballans | |
375 | 182 | Herve Ballans | h3. Automatic startup |
376 | 182 | Herve Ballans | |
377 | 182 | Herve Ballans | Add following lines in the /etc/rc.local file : |
378 | 182 | Herve Ballans | |
379 | 182 | Herve Ballans | <pre> |
380 | 197 | Herve Ballans | su -l -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D /var/lib/postgresql/data start" postgres |
381 | 197 | Herve Ballans | su -l -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D /var/lib/postgresql/data_sums start" postgres |
382 | 197 | Herve Ballans | su -l -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D /var/lib/postgresql/data_monitor start" postgres |
383 | 182 | Herve Ballans | </pre> |
384 | 182 | Herve Ballans | |
385 | 325 | Pablo Alingery | h3. Install additional modules |
386 | 327 | Pablo Alingery | |
387 | 326 | Pablo Alingery | We need dblink for triggers between db so let's install postgresql-contrib-9.4 |
388 | 325 | Pablo Alingery | |
389 | 325 | Pablo Alingery | <pre> |
390 | 326 | Pablo Alingery | root@sdo-new:~# apt-get install postgresql-contrib-9.4 |
391 | 325 | Pablo Alingery | </pre> |
392 | 325 | Pablo Alingery | |
393 | 328 | Pablo Alingery | Create extension dblink for each db |
394 | 328 | Pablo Alingery | ias_sdo |
395 | 325 | Pablo Alingery | <pre> |
396 | 328 | Pablo Alingery | root@sdo-new:$ psql ias_sdo -p5432 -U postgres |
397 | 328 | Pablo Alingery | psql (9.4.6) |
398 | 325 | Pablo Alingery | Type "help" for help. |
399 | 1 | Herve Ballans | |
400 | 328 | Pablo Alingery | ias_sdo=# CREATE EXTENSION dblink ; |
401 | 1 | Herve Ballans | CREATE EXTENSION |
402 | 328 | Pablo Alingery | </pre> |
403 | 1 | Herve Ballans | |
404 | 328 | Pablo Alingery | ias_sdo_sums |
405 | 328 | Pablo Alingery | <pre> |
406 | 328 | Pablo Alingery | root@sdo-new:$ psql ias_sdo -p5434 -U postgres |
407 | 328 | Pablo Alingery | psql (9.4.6) |
408 | 328 | Pablo Alingery | Type "help" for help. |
409 | 328 | Pablo Alingery | |
410 | 328 | Pablo Alingery | ias_sdo=# CREATE EXTENSION dblink ; |
411 | 328 | Pablo Alingery | CREATE EXTENSION |
412 | 328 | Pablo Alingery | </pre> |
413 | 328 | Pablo Alingery | |
414 | 328 | Pablo Alingery | ias_sdo_monitor |
415 | 328 | Pablo Alingery | <pre> |
416 | 328 | Pablo Alingery | root@sdo-new:$ psql ias_sdo -p5436 -U postgres |
417 | 328 | Pablo Alingery | psql (9.4.6) |
418 | 328 | Pablo Alingery | Type "help" for help. |
419 | 328 | Pablo Alingery | |
420 | 328 | Pablo Alingery | ias_sdo=# CREATE EXTENSION dblink ; |
421 | 328 | Pablo Alingery | CREATE EXTENSION |
422 | 325 | Pablo Alingery | </pre> |
423 | 325 | Pablo Alingery | |
424 | 330 | Pablo Alingery | To check the list of installed extensions type : |
425 | 330 | Pablo Alingery | <pre> |
426 | 330 | Pablo Alingery | ias_sdo=\dx |
427 | 332 | Herve Ballans | List of installed extensions |
428 | 332 | Herve Ballans | Name | Version | Schema | Description |
429 | 332 | Herve Ballans | ---------+---------+------------+-------------------------------------------------------------- |
430 | 332 | Herve Ballans | dblink | 1.1 | public | connect to other PostgreSQL databases from within a database |
431 | 332 | Herve Ballans | plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language |
432 | 332 | Herve Ballans | (2 rows) |
433 | 330 | Pablo Alingery | |
434 | 330 | Pablo Alingery | </pre> |
435 | 332 | Herve Ballans | |
436 | 332 | Herve Ballans | |
437 | 325 | Pablo Alingery | |
438 | 184 | Herve Ballans | h2. System tuning |
439 | 1 | Herve Ballans | |
440 | 187 | Herve Ballans | Current configuration : |
441 | 185 | Herve Ballans | <pre> |
442 | 185 | Herve Ballans | # sysctl -a |
443 | 185 | Herve Ballans | </pre> |
444 | 1 | Herve Ballans | |
445 | 187 | Herve Ballans | In the following file : |
446 | 185 | Herve Ballans | <pre> |
447 | 185 | Herve Ballans | # vi /etc/sysctl.conf |
448 | 185 | Herve Ballans | </pre> |
449 | 1 | Herve Ballans | |
450 | 188 | Herve Ballans | we change some values (default values are commented with '#') |
451 | 194 | Herve Ballans | |
452 | 195 | Herve Ballans | (resources links [[Links|here]]) |
453 | 194 | Herve Ballans | |
454 | 185 | Herve Ballans | <pre> |
455 | 191 | Herve Ballans | # Semaphore is a object that is used to control utilization of a particular process. |
456 | 191 | Herve Ballans | # kernel.sem = SEMMSL SEMMNS SEMOPM SEMMNI |
457 | 191 | Herve Ballans | # SEMMSL maximum number of semaphores per array (min 128) |
458 | 191 | Herve Ballans | # SEMMNS maximum semaphores system-wide |
459 | 191 | Herve Ballans | # SEMOPM maximum operations per semop call |
460 | 191 | Herve Ballans | # SEMMNI maximum arrays |
461 | 191 | Herve Ballans | #semop incresed from 32 (default value)operation to 100 (3rd value ) |
462 | 191 | Herve Ballans | # kernel.sem = 250 32000 32 128 |
463 | 191 | Herve Ballans | kernel.sem = 250 32000 100 128 |
464 | 191 | Herve Ballans | |
465 | 191 | Herve Ballans | In order to apply changes :#This sets the OS receive buffer size for all types of connections (default value 212992) |
466 | 191 | Herve Ballans | #net.core.rmem_default = 212992 |
467 | 191 | Herve Ballans | #net.core.rmem_max = 212992 |
468 | 191 | Herve Ballans | net.core.rmem_default = 33554432 |
469 | 191 | Herve Ballans | net.core.rmem_max = 33554432 |
470 | 191 | Herve Ballans | |
471 | 191 | Herve Ballans | #This is the OS send buffer size for all types of connections (212992 default value) |
472 | 191 | Herve Ballans | #net.core.wmem_default = 212992 |
473 | 191 | Herve Ballans | #net.core.wmem_max = 212992 |
474 | 191 | Herve Ballans | net.core.wmem_default = 33554432 |
475 | 191 | Herve Ballans | net.core.wmem_max = 33554432 |
476 | 191 | Herve Ballans | |
477 | 193 | Pablo Alingery | #TCP Autotuning setting. "The first value tells the kernel the minimum receive buffer for each TCP connection, and this buffer is always |
478 | 193 | Pablo Alingery | # allocated to a TCP socket, even under high pressure on the system. ... The second value specified tells the kernel the default receive |
479 | 193 | Pablo Alingery | # buffer allocated for each TCP socket. This value overrides the /proc/sys/net/core/rmem_default value used by other protocols. ... |
480 | 192 | Pablo Alingery | # The third and last value specified in this variable specifies the maximum receive buffer that can be allocated for a TCP socket." |
481 | 1 | Herve Ballans | #net.ipv4.tcp_rmem = 4096 87380 6291456 |
482 | 1 | Herve Ballans | net.ipv4.tcp_rmem = 10240 87380 33554432 |
483 | 1 | Herve Ballans | |
484 | 193 | Pablo Alingery | #TCP Autotuning setting. "This variable takes 3 different values which holds information on how much TCP sendbuffer memory space |
485 | 193 | Pablo Alingery | # each TCP socket has to use. Every TCP socket has this much buffer space to use before the buffer is filled up. Each of the three |
486 | 193 | Pablo Alingery | #values are used under different conditions. ... The first value in this variable tells the minimum TCP send buffer space available |
487 | 193 | Pablo Alingery | # for a single TCP socket. ... The second value in the variable tells us the default buffer space allowed for a single TCP socket to use. ... |
488 | 193 | Pablo Alingery | #The third value tells the kernel the maximum TCP send buffer space." |
489 | 191 | Herve Ballans | #net.ipv4.tcp_wmem = 4096 16384 4194304 |
490 | 191 | Herve Ballans | net.ipv4.tcp_wmem = 10240 87380 33554432 |
491 | 191 | Herve Ballans | |
492 | 191 | Herve Ballans | #Disable cache metrics so the initial conditions of the closed connections will not be saved to be used in near future connections |
493 | 191 | Herve Ballans | #net.ipv4.tcp_no_metrics_save = 0 |
494 | 191 | Herve Ballans | net.ipv4.tcp_no_metrics_save = 1 |
495 | 191 | Herve Ballans | |
496 | 1 | Herve Ballans | # Increase number of incoming connections backlog |
497 | 1 | Herve Ballans | # net.core.netdev_max_backlog = 1000 |
498 | 1 | Herve Ballans | net.core.netdev_max_backlog = 5000 |
499 | 1 | Herve Ballans | |
500 | 1 | Herve Ballans | # The tcp_mem variable defines how the TCP stack should behave when it comes to memory usage. ... |
501 | 192 | Pablo Alingery | #The first value specified in the tcp_mem variable tells the kernel the low threshold. |
502 | 192 | Pablo Alingery | #Below this point, the TCP stack do not bother at all about putting any pressure on the memory usage by different TCP sockets. ... |
503 | 193 | Pablo Alingery | #The second value tells the kernel at which point to start pressuring memory usage down. ... The final value tells the kernel how many |
504 | 193 | Pablo Alingery | # memory pages it may use maximally |
505 | 1 | Herve Ballans | # net.ipv4.tcp_mem = 1543347 2057796 3086694 |
506 | 191 | Herve Ballans | net.ipv4.tcp_mem = 786432 1048576 26777216 |
507 | 191 | Herve Ballans | |
508 | 192 | Pablo Alingery | # local port range that is used by TCP and UDP traffic to choose the local port. You will see in the parameters of this file two numbers: |
509 | 192 | Pablo Alingery | # The first number is the first local port allowed for TCP and UDP traffic on the server, the second is the last local port number. |
510 | 191 | Herve Ballans | # net.ipv4.ip_local_port_range = 32768 61000 |
511 | 191 | Herve Ballans | net.ipv4.ip_local_port_range = 1024 65535 |
512 | 191 | Herve Ballans | |
513 | 191 | Herve Ballans | # maximum number of sockets in TIME-WAIT to be held simultaneously. |
514 | 191 | Herve Ballans | # net.ipv4.tcp_max_tw_buckets = 262144 |
515 | 191 | Herve Ballans | net.ipv4.tcp_max_tw_buckets = 360000 |
516 | 1 | Herve Ballans | </pre> |
517 | 185 | Herve Ballans | |
518 | 191 | Herve Ballans | Then we run the following command to make our change take effect: |
519 | 191 | Herve Ballans | |
520 | 185 | Herve Ballans | <pre> |
521 | 185 | Herve Ballans | # sysctl -p |
522 | 185 | Herve Ballans | </pre> |
523 | 176 | Pablo Alingery | |
524 | 216 | Pablo Alingery | h2. JMD Installation |
525 | 215 | Herve Ballans | |
526 | 217 | Pablo Alingery | Download new JMD package build by Niles Oien 2016-04-07 following thoses actions : |
527 | 215 | Herve Ballans | |
528 | 215 | Herve Ballans | <pre> |
529 | 215 | Herve Ballans | cd /urs/local |
530 | 215 | Herve Ballans | |
531 | 215 | Herve Ballans | [oien@spsc-nso19-12 ~]$ ftp gong2.nso.edu |
532 | 215 | Herve Ballans | Name (gong2.nso.edu:oien): anonymous |
533 | 215 | Herve Ballans | Password: <---------------- use your email as the password |
534 | 215 | Herve Ballans | ftp> cd outgoing/oien |
535 | 215 | Herve Ballans | ftp> get pablo_jmd.tar.gz |
536 | 215 | Herve Ballans | ftp> quit |
537 | 215 | Herve Ballans | </pre> |
538 | 215 | Herve Ballans | |
539 | 215 | Herve Ballans | Install new JMD package following thoses actions |
540 | 215 | Herve Ballans | |
541 | 215 | Herve Ballans | <pre> |
542 | 215 | Herve Ballans | [oien@spsc-nso19-12 ~]$ gunzip -vf pablo_jmd.tar.gz |
543 | 215 | Herve Ballans | pablo_jmd.tar.gz: 8.3% -- replaced with pablo_jmd.tar |
544 | 215 | Herve Ballans | [oien@spsc-nso19-12 ~]$ tar xf pablo_jmd.tar |
545 | 215 | Herve Ballans | [oien@spsc-nso19-12 ~]$ cd pablo_jmd |
546 | 338 | Pablo Alingery | [oien@spsc-nso19-12 pablo_jmd]$ less README.txt |
547 | 215 | Herve Ballans | cd .. |
548 | 215 | Herve Ballans | mv jmd jmd_old |
549 | 215 | Herve Ballans | mv pablo_jmd jmd |
550 | 215 | Herve Ballans | </pre> |
551 | 215 | Herve Ballans | |
552 | 340 | Pablo Alingery | url provided by jsoc to download last JMD version http://vso.tuc.noao.edu/VSO/downloads/JMD |
553 | 339 | Pablo Alingery | |
554 | 266 | Herve Ballans | h2. Apache Configuration for JMD |
555 | 215 | Herve Ballans | |
556 | 215 | Herve Ballans | Configure webserver to request jsoc_fetch |
557 | 215 | Herve Ballans | (following the mail from nilesoien@gmail.com) |
558 | 215 | Herve Ballans | |
559 | 222 | Pablo Alingery | %{color:red}WARNING : change hostname when migrate sdo-new to sdo% |
560 | 222 | Pablo Alingery | |
561 | 220 | Pablo Alingery | config d 'apache2 fichier sdo-new.ias.u-psud.fr |
562 | 215 | Herve Ballans | <pre> |
563 | 215 | Herve Ballans | |
564 | 215 | Herve Ballans | <VirtualHost *:80> |
565 | 220 | Pablo Alingery | ServerName sdo-new.ias.u-psud.fr |
566 | 215 | Herve Ballans | ServerAdmin pablo.alingery@ias.u-psud.fr |
567 | 215 | Herve Ballans | #ErrorLog /var/log/apache2/error.log |
568 | 215 | Herve Ballans | DocumentRoot /var/www |
569 | 215 | Herve Ballans | |
570 | 215 | Herve Ballans | # Possible values include: debug, info, notice, warn, error, crit, |
571 | 215 | Herve Ballans | # alert, emerg. |
572 | 215 | Herve Ballans | LogLevel debug |
573 | 215 | Herve Ballans | |
574 | 215 | Herve Ballans | #CustomLog /var/log/apache2/VSO/access.log combined |
575 | 215 | Herve Ballans | |
576 | 215 | Herve Ballans | Alias /VSO/DRMS/cgi-bin/ "/home/production/netdrms-tools/scripts/cgi-bin/" |
577 | 215 | Herve Ballans | |
578 | 215 | Herve Ballans | <Location /VSO/DRMS/cgi-bin> |
579 | 215 | Herve Ballans | SetHandler perl-script |
580 | 215 | Herve Ballans | PerlResponseHandler ModPerl::Registry |
581 | 215 | Herve Ballans | PerlOptions +ParseHeaders |
582 | 215 | Herve Ballans | Options +ExecCGI |
583 | 215 | Herve Ballans | Order allow,deny |
584 | 215 | Herve Ballans | Allow from all |
585 | 215 | Herve Ballans | </Location> |
586 | 215 | Herve Ballans | </VirtualHost> |
587 | 215 | Herve Ballans | |
588 | 215 | Herve Ballans | </pre> |
589 | 215 | Herve Ballans | |
590 | 223 | Herve Ballans | Activate the new virtualhost and disable the default one : |
591 | 223 | Herve Ballans | <pre> |
592 | 225 | Herve Ballans | # a2ensite sdo-new.ias.u-psud.fr.conf |
593 | 225 | Herve Ballans | # a2dissite 000-default.conf |
594 | 225 | Herve Ballans | # service apache2 restart |
595 | 225 | Herve Ballans | </pre> |
596 | 225 | Herve Ballans | |
597 | 225 | Herve Ballans | Allow symlink for the cgi scripts dierctory in apache2.conf : |
598 | 225 | Herve Ballans | <pre> |
599 | 225 | Herve Ballans | <Directory /home/production/netdrms-tools/scripts/> |
600 | 225 | Herve Ballans | Options FollowSymLinks |
601 | 225 | Herve Ballans | AllowOverride None |
602 | 225 | Herve Ballans | Require all granted |
603 | 225 | Herve Ballans | </Directory> |
604 | 225 | Herve Ballans | </pre> |
605 | 225 | Herve Ballans | |
606 | 225 | Herve Ballans | <pre> |
607 | 225 | Herve Ballans | # service apache2 restart |
608 | 223 | Herve Ballans | </pre> |
609 | 223 | Herve Ballans | |
610 | 226 | Pablo Alingery | %{color:red}To be added futher to test cgi% |
611 | 1 | Herve Ballans | Execution test cgi with URL : |
612 | 226 | Pablo Alingery | |
613 | 226 | Pablo Alingery | make sure my $hostname=in the cgi file vso_jsoc_fetch.cgi is correct and test |
614 | 224 | Pablo Alingery | http://sdo-new.ias.u-psud.fr/VSO/DRMS/cgi-bin/vso_jsoc_fetch.cgi |
615 | 215 | Herve Ballans | |
616 | 215 | Herve Ballans | expected result : |
617 | 215 | Herve Ballans | <pre> |
618 | 215 | Herve Ballans | { |
619 | 215 | Herve Ballans | "wait" : 0, |
620 | 215 | Herve Ballans | "requestid" : "", |
621 | 215 | Herve Ballans | "data" : {}, |
622 | 215 | Herve Ballans | "method" : "url_quick", |
623 | 215 | Herve Ballans | "size" : 0, |
624 | 215 | Herve Ballans | "errormsg" : "Empty query", |
625 | 215 | Herve Ballans | "protocol" : "as-is", |
626 | 215 | Herve Ballans | "status" : 1, |
627 | 215 | Herve Ballans | "space_ratio" : 0.1, |
628 | 215 | Herve Ballans | "load_ratio" : 0.066875, |
629 | 215 | Herve Ballans | "dir" : "", |
630 | 215 | Herve Ballans | "count" : 0 |
631 | 215 | Herve Ballans | } |
632 | 215 | Herve Ballans | </pre> |
633 | 215 | Herve Ballans | |
634 | 267 | Herve Ballans | h2. JMD configuration |
635 | 267 | Herve Ballans | |
636 | 291 | Herve Ballans | Modify the file [[/usr/local/jmd/cfg/JMD.cfg]] |
637 | 267 | Herve Ballans | |
638 | 267 | Herve Ballans | |
639 | 271 | Herve Ballans | %{color:red}Warning : when migrate production server, change _sdo-new_ by _sdo_% |
640 | 271 | Herve Ballans | |
641 | 1 | Herve Ballans | h2. NetDRMS tool Install |
642 | 23 | Pablo Alingery | |
643 | 199 | Pablo Alingery | The following installed is based on : [[http://inf-redmine.ias.u-psud.fr/redmine/attachments/download/114/Netdrms_First_Install.odt]] |
644 | 1 | Herve Ballans | |
645 | 207 | Pablo Alingery | The sources are in : http://jsoc.stanford.edu/netdrms/dist/ |
646 | 207 | Pablo Alingery | Locally on sdo-new in : /home/production/sources |
647 | 1 | Herve Ballans | |
648 | 199 | Pablo Alingery | Symbolic link created 'netdrms_current' that corresponds to the last release |
649 | 350 | Pablo Alingery | <pre> |
650 | 350 | Pablo Alingery | ln -s netdrms_8.12/ netdrms_current |
651 | 350 | Pablo Alingery | </pre> |
652 | 360 | Pablo Alingery | Edit file config.local and make sure that resource exist |
653 | 360 | Pablo Alingery | |
654 | 360 | Pablo Alingery | for example : |
655 | 360 | Pablo Alingery | <pre> |
656 | 360 | Pablo Alingery | # defaults |
657 | 360 | Pablo Alingery | # PostgreSQL API headers (must contain libpq-fe.h) |
658 | 360 | Pablo Alingery | POSTGRES_INCS:X86_64 /usr/include/postgresql |
659 | 360 | Pablo Alingery | POSTGRES_INCS:AVX /usr/include/postgresql |
660 | 360 | Pablo Alingery | |
661 | 360 | Pablo Alingery | # the location of the PostgreSQL libs; likely to be either |
662 | 360 | Pablo Alingery | # /usr/lib or /usr/lib64 or /usr/local/pgsql/lib |
663 | 360 | Pablo Alingery | POSTGRES_LIBS:X86_64 /usr/lib/postgresql/9.4/lib |
664 | 360 | Pablo Alingery | POSTGRES_LIBS:AVX /usr/lib/postgresql/9.4/lib |
665 | 360 | Pablo Alingery | </pre> |
666 | 360 | Pablo Alingery | |
667 | 360 | Pablo Alingery | So make sure /usr/include/postgresql does exist and is the location of libpq-fe.h for example (see in config.local comments about POSTGRES API headers ) |
668 | 360 | Pablo Alingery | |
669 | 27 | Pablo Alingery | <pre> |
670 | 1 | Herve Ballans | cp config.local.template config.local |
671 | 1 | Herve Ballans | </pre> |
672 | 29 | Pablo Alingery | |
673 | 350 | Pablo Alingery | %{color:red}Warning WEB_DBUSER apache we think that it should value www maybe , to be checked testing the cgi% |
674 | 1 | Herve Ballans | %{color:red}So check the cgi to figure out which user is used to connect to bd% |
675 | 32 | Herve Ballans | |
676 | 199 | Pablo Alingery | Create 2 followings dirs : |
677 | 1 | Herve Ballans | <pre> |
678 | 350 | Pablo Alingery | mkdir /usr/local/netdrms_8.12/logs/SUM -p |
679 | 350 | Pablo Alingery | mkdir /usr/local/netdrms_8.12/logs/slony -p |
680 | 27 | Pablo Alingery | </pre> |
681 | 1 | Herve Ballans | |
682 | 205 | Pablo Alingery | %{color:red}Warning config.local from line 150 =>222 Remote config to provide data, series to externals% |
683 | 205 | Pablo Alingery | %{color:red}To be checked for PLATO% |
684 | 1 | Herve Ballans | |
685 | 1 | Herve Ballans | Création d'un alias netdrms dans /root/.bashrc |
686 | 27 | Pablo Alingery | <pre> |
687 | 1 | Herve Ballans | alias netdrms='cd /usr/local/netdrms_current' |
688 | 1 | Herve Ballans | </pre> |
689 | 31 | Pablo Alingery | |
690 | 1 | Herve Ballans | h3. Prepare compiling |
691 | 1 | Herve Ballans | |
692 | 1 | Herve Ballans | Into _production_ profile |
693 | 352 | Pablo Alingery | Replace all " $? !=0 " by " $status !=0 " in file configure then tape |
694 | 246 | Herve Ballans | <pre> |
695 | 252 | Herve Ballans | $ ./configure |
696 | 246 | Herve Ballans | </pre> |
697 | 246 | Herve Ballans | |
698 | 364 | Pablo Alingery | Again in netdrms_9.0 |
699 | 347 | Pablo Alingery | |
700 | 358 | Pablo Alingery | Modification du fichier /usr/local/netdrms_current/build/jsoc_machine.csh (Same issue in NetDrms 8.11 ) |
701 | 347 | Pablo Alingery | |
702 | 347 | Pablo Alingery | ligne 28 : |
703 | 347 | Pablo Alingery | |
704 | 347 | Pablo Alingery | case "x86_64": |
705 | 347 | Pablo Alingery | echo linux_x86_64 |
706 | 347 | Pablo Alingery | breaksw |
707 | 364 | Pablo Alingery | |
708 | 364 | Pablo Alingery | Again in netdrms_9.0 |
709 | 347 | Pablo Alingery | |
710 | 347 | Pablo Alingery | Overwriting the existing files in the right place (base/sums/libs/pg) & modif in jsoc_machine.csh |
711 | 347 | Pablo Alingery | => the command 'make' works well. |
712 | 347 | Pablo Alingery | |
713 | 113 | Pablo Alingery | h3. Compilation |
714 | 113 | Pablo Alingery | |
715 | 361 | Pablo Alingery | In file make_basic.mk line 223 |
716 | 359 | Pablo Alingery | look for "GCC_LF_ALL =" |
717 | 361 | Pablo Alingery | add "-lcrypto" |
718 | 353 | Pablo Alingery | |
719 | 353 | Pablo Alingery | |
720 | 249 | Herve Ballans | <pre> |
721 | 253 | Herve Ballans | $ make |
722 | 253 | Herve Ballans | $ make sums |
723 | 113 | Pablo Alingery | </pre> |
724 | 113 | Pablo Alingery | |
725 | 255 | Herve Ballans | h2. SSH-HPN install |
726 | 255 | Herve Ballans | |
727 | 258 | Herve Ballans | Intallation de openssh 7.2p2 car le dernier patch hpn dispo est 7.2hpn14.V10 |
728 | 255 | Herve Ballans | |
729 | 258 | Herve Ballans | web site for open ssh : http://ftp2.fr.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-7.2p2.tar.gz |
730 | 258 | Herve Ballans | patch hpn : https://sourceforge.net/projects/hpnssh/files/HPN-SSH%2014v10%207.2p2/openssh-7_2_P2-hpn-14.10.diff |
731 | 255 | Herve Ballans | |
732 | 256 | Pablo Alingery | In the dir /home/production/sources into 'root' |
733 | 255 | Herve Ballans | |
734 | 255 | Herve Ballans | <pre> |
735 | 258 | Herve Ballans | # tar -xzvf openssh-7.2p2.tar.gz |
736 | 258 | Herve Ballans | # cd openssh-7.2p2/ |
737 | 258 | Herve Ballans | # cat ../openssh-7_2_P2-hpn-14.10.diff |patch -p1 |
738 | 255 | Herve Ballans | # ./configure --prefix=/usr/local/hpn-ssh --with-pam --with-md5-passwords --without-zlib-version-check --with-tcp-wrappers |
739 | 255 | Herve Ballans | # make |
740 | 255 | Herve Ballans | # make install |
741 | 255 | Herve Ballans | </pre> |
742 | 255 | Herve Ballans | |
743 | 255 | Herve Ballans | Depuis sdo : |
744 | 255 | Herve Ballans | <pre> |
745 | 255 | Herve Ballans | # cd /usr/local/hpn-ssh/etc/ |
746 | 259 | Herve Ballans | # rsync -av *key* production@sdo-new:/home/production/ |
747 | 255 | Herve Ballans | </pre> |
748 | 255 | Herve Ballans | |
749 | 273 | Herve Ballans | Following the link : http://vso.stanford.edu/netdrms/rmtsums.html |
750 | 1 | Herve Ballans | <pre> |
751 | 274 | Herve Ballans | $ cd /home/production/ |
752 | 273 | Herve Ballans | $ ssh-agent -c > ~/.ssh-agent_rs |
753 | 285 | Pablo Alingery | </pre> |
754 | 363 | Pablo Alingery | NB : correct file ~/.ssh-agent_rs replace 'setenv' by 'export' and add '=' between var and value |
755 | 285 | Pablo Alingery | <pre> |
756 | 285 | Pablo Alingery | #!/bin/csh |
757 | 285 | Pablo Alingery | export SSH_AUTH_SOCK=/tmp/ssh-9POrTXobhLR4/agent.74272; |
758 | 285 | Pablo Alingery | export SSH_AGENT_PID=74273; |
759 | 285 | Pablo Alingery | echo Agent pid 74273; |
760 | 285 | Pablo Alingery | </pre> |
761 | 285 | Pablo Alingery | |
762 | 285 | Pablo Alingery | <pre> |
763 | 273 | Herve Ballans | $ source ~/.ssh-agent_rs |
764 | 273 | Herve Ballans | $ ssh-add ~/.ssh/id_rsa |
765 | 269 | Herve Ballans | </pre> |
766 | 269 | Herve Ballans | |
767 | 259 | Herve Ballans | Sur sdo-new : |
768 | 255 | Herve Ballans | <pre> |
769 | 260 | Herve Ballans | # mv /home/production/*key* /usr/local/hpn-ssh/etc/ |
770 | 255 | Herve Ballans | </pre> |
771 | 255 | Herve Ballans | |
772 | 255 | Herve Ballans | We have to configure the port number to 55000 |
773 | 255 | Herve Ballans | <pre> |
774 | 255 | Herve Ballans | # vi ssh_config |
775 | 255 | Herve Ballans | </pre> |
776 | 255 | Herve Ballans | |
777 | 255 | Herve Ballans | and add : |
778 | 255 | Herve Ballans | <pre> |
779 | 255 | Herve Ballans | #Port 22 |
780 | 255 | Herve Ballans | Port 55000 |
781 | 255 | Herve Ballans | </pre> |
782 | 255 | Herve Ballans | |
783 | 255 | Herve Ballans | We configure also the server hpn (even we don't use it yet) |
784 | 255 | Herve Ballans | <pre> |
785 | 255 | Herve Ballans | # vi sshd_config |
786 | 255 | Herve Ballans | </pre> |
787 | 255 | Herve Ballans | |
788 | 255 | Herve Ballans | and add : |
789 | 255 | Herve Ballans | <pre> |
790 | 255 | Herve Ballans | #Port 22 |
791 | 255 | Herve Ballans | Port 55000 |
792 | 255 | Herve Ballans | |
793 | 255 | Herve Ballans | #PidFile /var/run/sshd.pid |
794 | 255 | Herve Ballans | PidFile /var/run/sshd.55000.pid |
795 | 255 | Herve Ballans | |
796 | 255 | Herve Ballans | # allow the use of the none cipher |
797 | 255 | Herve Ballans | #NoneEnabled no |
798 | 255 | Herve Ballans | NoneEnabled yes |
799 | 255 | Herve Ballans | </pre> |
800 | 255 | Herve Ballans | |
801 | 265 | Herve Ballans | In order to force the use of SSH-hpn by JMD : |
802 | 265 | Herve Ballans | |
803 | 265 | Herve Ballans | <pre> |
804 | 265 | Herve Ballans | $ vi /usr/local/jmd/cfg/JMD.cfg |
805 | 265 | Herve Ballans | </pre> |
806 | 265 | Herve Ballans | |
807 | 265 | Herve Ballans | and replace the path of scp binary : |
808 | 265 | Herve Ballans | |
809 | 265 | Herve Ballans | <pre> |
810 | 265 | Herve Ballans | SCPCommand=/usr/local/hpn-ssh/bin/scp -o NoneSwitch=yes -o NoneEnabled=yes |
811 | 265 | Herve Ballans | </pre> |
812 | 265 | Herve Ballans | |
813 | 262 | Herve Ballans | h2. Private key Public key generation |
814 | 262 | Herve Ballans | |
815 | 262 | Herve Ballans | JSOC need your public key to directly connect to your netdrms server. In order to generate your RSA public/private key pair : |
816 | 262 | Herve Ballans | |
817 | 262 | Herve Ballans | <pre> |
818 | 262 | Herve Ballans | $ ssh-keygen -t rsa |
819 | 262 | Herve Ballans | </pre> |
820 | 262 | Herve Ballans | |
821 | 263 | Pablo Alingery | Not necessary in our case cause we recover the ssh keys of the current server 'sdo'. |
822 | 262 | Herve Ballans | |
823 | 262 | Herve Ballans | However, we have to retrieve the private key of production account on sdo-new |
824 | 262 | Herve Ballans | |
825 | 262 | Herve Ballans | From sdo : |
826 | 262 | Herve Ballans | <pre> |
827 | 262 | Herve Ballans | rsync -av id_rsa production@sdo-new:/home/production/.ssh/ |
828 | 262 | Herve Ballans | </pre> |
829 | 262 | Herve Ballans | |
830 | 300 | Herve Ballans | In order to test the connection : |
831 | 300 | Herve Ballans | <pre> |
832 | 300 | Herve Ballans | $ ssh jsocexp@jsocport.stanford.edu -p55000 |
833 | 300 | Herve Ballans | </pre> |
834 | 300 | Herve Ballans | |
835 | 301 | Herve Ballans | Note : the IP of the server has to be declared to the jsoc |
836 | 301 | Herve Ballans | |
837 | 299 | Pablo Alingery | h2. NetDRMS Replication Config |
838 | 112 | Herve Ballans | |
839 | 319 | Herve Ballans | %{color:red}To be check if we keep that here or not% |
840 | 311 | Pablo Alingery | To prevent from deleting files |
841 | 307 | Pablo Alingery | <pre> |
842 | 307 | Pablo Alingery | cd logs/SUM ; scp production@sdo:/usr/local/netdrms/install-config-files/sum_rm.cfg . |
843 | 112 | Herve Ballans | </pre> |
844 | 1 | Herve Ballans | |
845 | 1 | Herve Ballans | Create tmp dir in netdrms working directory (/usr/local/netdrms_current) |
846 | 112 | Herve Ballans | <pre> |
847 | 315 | Herve Ballans | $ mkdir tmp |
848 | 112 | Herve Ballans | </pre> |
849 | 112 | Herve Ballans | |
850 | 308 | Pablo Alingery | Create config file from template |
851 | 112 | Herve Ballans | <pre> |
852 | 315 | Herve Ballans | $ cd base/drms/replication/etc |
853 | 313 | Pablo Alingery | cp repclient.template.cfg ias.repclient.cfg |
854 | 307 | Pablo Alingery | </pre> |
855 | 91 | Herve Ballans | |
856 | 316 | Herve Ballans | Define node in file : [[/usr/local/netdrms_current/base/drms/replication/etc/ias.repclient.cfg]] |
857 | 1 | Herve Ballans | |
858 | 309 | Pablo Alingery | In our case : |
859 | 94 | Herve Ballans | <pre> |
860 | 1 | Herve Ballans | node=IASprod |
861 | 319 | Herve Ballans | </pre> |
862 | 319 | Herve Ballans | |
863 | 341 | Pablo Alingery | There are 4 new fields in the new version of repclient (see Art mail June 11, 2016) |
864 | 335 | Herve Ballans | |
865 | 319 | Herve Ballans | <pre> |
866 | 320 | Herve Ballans | #kRSBaseURL=<base URL for all Slony services> |
867 | 334 | Herve Ballans | kRSBaseURL=http://jsoc.stanford.edu/cgi-bin/ajax |
868 | 320 | Herve Ballans | #kSubService=<URL of the subscription service> |
869 | 334 | Herve Ballans | kSubService=${kRSBaseURL}/request-subs.py |
870 | 320 | Herve Ballans | #kPubListService=<URL of the publication-list service> |
871 | 334 | Herve Ballans | kPubListService=${kRSBaseURL}/publist.py |
872 | 320 | Herve Ballans | #kSubXfer=<URL of the file transfer directory> |
873 | 334 | Herve Ballans | kSubXfer=http://jsoc.stanford.edu/subscription |
874 | 106 | Pablo Alingery | </pre> |
875 | 321 | Pablo Alingery | |
876 | 321 | Pablo Alingery | h2. NetDRMS Subsribe list Config |
877 | 321 | Pablo Alingery | |
878 | 321 | Pablo Alingery | Create config subscribe file from template |
879 | 321 | Pablo Alingery | <pre> |
880 | 321 | Pablo Alingery | $ cd base/drms/replication/etc |
881 | 321 | Pablo Alingery | cp subscribe_list.template.cfg ias.subscribe_list.cfg |
882 | 321 | Pablo Alingery | </pre> |
883 | 321 | Pablo Alingery | |
884 | 292 | Pablo Alingery | h2. TO BE DONE BEFORE MIGRATION |
885 | 1 | Herve Ballans | |
886 | 152 | Pablo Alingery | %{color:red} configure sytem taille buffer% |
887 | 152 | Pablo Alingery | |
888 | 155 | Pablo Alingery | %{color:red} swap désactivé ? discussion Stephane et Gilles% |
889 | 1 | Herve Ballans | |
890 | 1 | Herve Ballans | %{color:red}Test ssh connection to JSOC , NSO CFA before subcription and just after name sdo-new into sdo% |
891 | 317 | Pablo Alingery | %{color:red}Take old counter and place it in the same dir before migration% |
892 | 354 | Pablo Alingery | |
893 | 354 | Pablo Alingery | %{color:red}Before import existing db into postgres dont't forget to drop old one% |
894 | 356 | Pablo Alingery | |
895 | 357 | Pablo Alingery | %{color:red}Install VSO, test cgi to access hmi/aia data% |