NetDRMS Installation » History » Version 365
Pablo Alingery, 12/09/2018 17:11
1 | 50 | Herve Ballans | {{toc}} |
---|---|---|---|
2 | 1 | Herve Ballans | |
3 | 351 | Pablo Alingery | h1. NetDRMS 8.12 Installation |
4 | 1 | Herve Ballans | |
5 | 276 | Herve Ballans | h2. Preliminary note |
6 | 275 | Herve Ballans | |
7 | 275 | Herve Ballans | In the following sections, when there is a command line : |
8 | 324 | Herve Ballans | <pre>#</pre>means that command is executed into _root_ user |
9 | 324 | Herve Ballans | <pre>$</pre>means that command is executed into _production_ user |
10 | 275 | Herve Ballans | |
11 | 333 | Herve Ballans | h2. Environment |
12 | 333 | Herve Ballans | |
13 | 343 | Herve Ballans | Server : DELL PowerEdge R730 |
14 | 343 | Herve Ballans | RAM : 64.00GB |
15 | 343 | Herve Ballans | CPU : 32 CPUs Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz |
16 | 345 | Herve Ballans | |
17 | 345 | Herve Ballans | Storage Configuration : |
18 | 344 | Herve Ballans | * 264 GB mounted in / (2 X 278.88 GB SAS 12G disks in RAID1) |
19 | 344 | Herve Ballans | * 1,7 TB mounted in /var/lib/postgresql (7 X 278.88 GB SAS 12G disks in RAID5 configuration) |
20 | 344 | Herve Ballans | * 1 global spare for both RAID |
21 | 344 | Herve Ballans | * DDP storage mounted in /SUM* |
22 | 343 | Herve Ballans | |
23 | 343 | Herve Ballans | Guest OS : Debian 8.4 |
24 | 343 | Herve Ballans | Kernel : Linux sdo-new 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-1 (2016-03-06) x86_64 GNU/Linux |
25 | 343 | Herve Ballans | |
26 | 1 | Herve Ballans | h2. Requirements |
27 | 1 | Herve Ballans | |
28 | 47 | Pablo Alingery | h3. Configuring production environment |
29 | 1 | Herve Ballans | |
30 | 284 | Herve Ballans | Adding user production |
31 | 47 | Pablo Alingery | |
32 | 1 | Herve Ballans | <pre> |
33 | 1 | Herve Ballans | # adduser production |
34 | 1 | Herve Ballans | # addgroup ias |
35 | 1 | Herve Ballans | # adduser production ias |
36 | 1 | Herve Ballans | </pre> |
37 | 1 | Herve Ballans | |
38 | 49 | Herve Ballans | h3. Configure /home/production/.profile |
39 | 1 | Herve Ballans | |
40 | 49 | Herve Ballans | Adding environment variables |
41 | 1 | Herve Ballans | |
42 | 1 | Herve Ballans | <pre> |
43 | 122 | Herve Ballans | #Modif Herve.B Pablo.A 12/05/2016 |
44 | 122 | Herve Ballans | |
45 | 1 | Herve Ballans | PATH="/usr/local/netdrms_current/bin/linux_x86_64:$PATH" |
46 | 49 | Herve Ballans | PATH="/usr/local/netdrms_current/scripts:$PATH" |
47 | 122 | Herve Ballans | PATH="/usr/local/jmd/bin:$PATH" |
48 | 122 | Herve Ballans | PATH="/usr/local/jmd/scripts:$PATH" |
49 | 141 | Pablo Alingery | #The following is not needed as installed with 'apt-get install' |
50 | 141 | Pablo Alingery | #PATH="/usr/local/pgsql/bin:$PATH" |
51 | 122 | Herve Ballans | PATH="/usr/local/netdrms-tools/scripts:$PATH" |
52 | 49 | Herve Ballans | #2014-12-17 Avec Herve et Pablo on pense que la ligne suivante est intutile as it is f.. install in /usr/local/lib |
53 | 140 | Pablo Alingery | PATH="/usr/local/cfitsio:$PATH" |
54 | 1 | Herve Ballans | #Done |
55 | 49 | Herve Ballans | |
56 | 122 | Herve Ballans | DERBY_HOME=/usr/local/jmd/databases/derby/derbyBD |
57 | 49 | Herve Ballans | export DERBY_HOME |
58 | 49 | Herve Ballans | |
59 | 49 | Herve Ballans | export CVSROOT=:pserver:anonymous@solarch.tuc.noao.edu:2401/vtarc1/vso/cvsroot |
60 | 49 | Herve Ballans | </pre> |
61 | 49 | Herve Ballans | |
62 | 162 | Pablo Alingery | h3. Configure /varl/lib/postgres/.profile |
63 | 161 | Pablo Alingery | |
64 | 163 | Pablo Alingery | Add to $PATH so postgres can exeute command like initdb |
65 | 161 | Pablo Alingery | |
66 | 161 | Pablo Alingery | <pre> |
67 | 161 | Pablo Alingery | #Modif Herve.B Pablo.A 19/05/2016 |
68 | 161 | Pablo Alingery | PATH="/usr/lib/postgresql/9.4/bin:$PATH" |
69 | 161 | Pablo Alingery | </pre> |
70 | 161 | Pablo Alingery | |
71 | 125 | Pablo Alingery | h3. Setting privileges for SUMS_MANAGER |
72 | 87 | Herve Ballans | |
73 | 365 | Pablo Alingery | Please modify file /etc/sudoers so user 'production' will be able to execute 'sum_chown' that we will located in : /usr/local/bin . See below |
74 | 365 | Pablo Alingery | |
75 | 125 | Pablo Alingery | Add to /etc/sudoers the following line : |
76 | 1 | Herve Ballans | production host=NOPASSWD:/usr/local/bin/sum_chmown |
77 | 1 | Herve Ballans | |
78 | 127 | Pablo Alingery | h3. Set passwords file for db postgres |
79 | 1 | Herve Ballans | |
80 | 127 | Pablo Alingery | Please create file .pgpass with the following information for user 'production" so he will be able to connect without password as 'production' or 'postgres' to 2 db that we will create later ias_sdo and ias_sdo_sums. |
81 | 1 | Herve Ballans | |
82 | 1 | Herve Ballans | |
83 | 1 | Herve Ballans | .pgpass content : |
84 | 1 | Herve Ballans | <pre> |
85 | 128 | Pablo Alingery | #hostname:5432:ias_sdo:production: |
86 | 128 | Pablo Alingery | #hostname:5434:ias_sdo_sums:production: |
87 | 128 | Pablo Alingery | #hostname:5432:ias_sdo:postgres: |
88 | 128 | Pablo Alingery | #hostname:5434:ias_sdo_sums:postgres |
89 | 1 | Herve Ballans | </pre> |
90 | 128 | Pablo Alingery | |
91 | 129 | Pablo Alingery | where #hostname is the name of your server |
92 | 129 | Pablo Alingery | |
93 | 129 | Pablo Alingery | |
94 | 131 | Pablo Alingery | %{color:red}WARNING : change hostname when migrate sdo-new to sdo% |
95 | 1 | Herve Ballans | |
96 | 133 | Pablo Alingery | h3. Install missing libraries |
97 | 1 | Herve Ballans | |
98 | 293 | Herve Ballans | * libreadline-dev (but already installed with postgresql) |
99 | 293 | Herve Ballans | * libssl-dev |
100 | 293 | Herve Ballans | * libpam0g-dev |
101 | 293 | Herve Ballans | * libperl-dev |
102 | 293 | Herve Ballans | * libnet-ssh-perl |
103 | 293 | Herve Ballans | * libstring-shellquote-perl |
104 | 296 | Herve Ballans | * libdbi-perl |
105 | 296 | Herve Ballans | * libdbd-pg-perl |
106 | 1 | Herve Ballans | |
107 | 133 | Pablo Alingery | NB : Already done in Debian 8.4 Jessy |
108 | 1 | Herve Ballans | <pre> |
109 | 1 | Herve Ballans | # apt-get install $library_name |
110 | 1 | Herve Ballans | </pre> |
111 | 1 | Herve Ballans | where |
112 | 1 | Herve Ballans | _$library_name_ is the name of library |
113 | 19 | Pablo Alingery | |
114 | 1 | Herve Ballans | Then to update path on libraries, please type : |
115 | 1 | Herve Ballans | <pre> |
116 | 19 | Pablo Alingery | # ldconfig |
117 | 1 | Herve Ballans | </pre> |
118 | 1 | Herve Ballans | |
119 | 1 | Herve Ballans | h3. Reconfigure locale |
120 | 1 | Herve Ballans | |
121 | 1 | Herve Ballans | <pre> |
122 | 1 | Herve Ballans | # dpkg-reconfigure locales |
123 | 1 | Herve Ballans | </pre> |
124 | 1 | Herve Ballans | |
125 | 1 | Herve Ballans | select fr_FR.UTF8 UTF8 and unselect en_US.UTF-8 |
126 | 1 | Herve Ballans | |
127 | 1 | Herve Ballans | localedef -i en_US -f UTF-8 en_US.UTF-8 |
128 | 1 | Herve Ballans | |
129 | 1 | Herve Ballans | (see here : [[http://www.thomas-krenn.com/en/wiki/Perl_warning_Setting_locale_failed_in_Debian]]) |
130 | 1 | Herve Ballans | |
131 | 134 | Pablo Alingery | h3. Install gfortran packages |
132 | 1 | Herve Ballans | |
133 | 1 | Herve Ballans | <pre> |
134 | 1 | Herve Ballans | # apt-get install gfortran swig |
135 | 1 | Herve Ballans | </pre> |
136 | 1 | Herve Ballans | |
137 | 1 | Herve Ballans | h3. Installation of cfitsio library |
138 | 1 | Herve Ballans | |
139 | 135 | Pablo Alingery | In /usr/local dir |
140 | 135 | Pablo Alingery | |
141 | 135 | Pablo Alingery | Version 3.39 (currently on sdo : 3.24) |
142 | 1 | Herve Ballans | <pre> |
143 | 135 | Pablo Alingery | # wget ftp://heasarc.gsfc.nasa.gov/software/fitsio/c/cfitsio3390.tar.gz |
144 | 1 | Herve Ballans | </pre> |
145 | 137 | Pablo Alingery | Untar and decompression |
146 | 1 | Herve Ballans | <pre> |
147 | 136 | Pablo Alingery | # tar -xzvf cfitsio3390.tar.gz |
148 | 1 | Herve Ballans | </pre> |
149 | 1 | Herve Ballans | |
150 | 139 | Pablo Alingery | Go into /usr/local and type |
151 | 1 | Herve Ballans | |
152 | 1 | Herve Ballans | su - root and : |
153 | 1 | Herve Ballans | <pre> |
154 | 1 | Herve Ballans | # ./configure --prefix=/usr/local |
155 | 1 | Herve Ballans | # make |
156 | 1 | Herve Ballans | # make install |
157 | 135 | Pablo Alingery | </pre> |
158 | 118 | Pablo Alingery | |
159 | 237 | Herve Ballans | h3. Installation des libraries libtar-dev, libcurl3-dev et libecpg-dev |
160 | 237 | Herve Ballans | |
161 | 237 | Herve Ballans | <pre> |
162 | 237 | Herve Ballans | apt-get install libtar libtar-dev |
163 | 237 | Herve Ballans | </pre> |
164 | 237 | Herve Ballans | |
165 | 238 | Pablo Alingery | Note, selecting 'libcurl4-openssl-dev' instead of 'libcurl3-dev' |
166 | 238 | Pablo Alingery | libcurl4-openssl-dev is already the newest version |
167 | 237 | Herve Ballans | <pre> |
168 | 237 | Herve Ballans | apt-get install libcurl3-dev |
169 | 237 | Herve Ballans | </pre> |
170 | 237 | Herve Ballans | |
171 | 237 | Herve Ballans | <pre> |
172 | 237 | Herve Ballans | apt-get install libecpg-dev |
173 | 237 | Herve Ballans | </pre> |
174 | 237 | Herve Ballans | |
175 | 144 | Pablo Alingery | h3. Installation of apache2 and lib perl |
176 | 118 | Pablo Alingery | |
177 | 118 | Pablo Alingery | <pre> |
178 | 118 | Pablo Alingery | # apt-get install apache2 |
179 | 118 | Pablo Alingery | |
180 | 118 | Pablo Alingery | # apt-get install libjson-perl |
181 | 118 | Pablo Alingery | |
182 | 118 | Pablo Alingery | # apt-get install libapache2-mod-perl2 |
183 | 118 | Pablo Alingery | </pre> |
184 | 1 | Herve Ballans | |
185 | 183 | Herve Ballans | h3. Installation of postgresql |
186 | 1 | Herve Ballans | |
187 | 145 | Pablo Alingery | Into 'root' |
188 | 145 | Pablo Alingery | |
189 | 145 | Pablo Alingery | <pre> |
190 | 147 | Pablo Alingery | apt-get install postgresql-9.4 postgresql-client-9.4 |
191 | 145 | Pablo Alingery | </pre> |
192 | 145 | Pablo Alingery | |
193 | 213 | Herve Ballans | h3. Installation of python3 |
194 | 209 | Pablo Alingery | |
195 | 209 | Pablo Alingery | Into 'root' |
196 | 209 | Pablo Alingery | |
197 | 209 | Pablo Alingery | <pre> |
198 | 209 | Pablo Alingery | apt-get install python3 |
199 | 1 | Herve Ballans | </pre> |
200 | 210 | Pablo Alingery | |
201 | 211 | Pablo Alingery | Add some modules psycopg2 (postgres connector) and pySmartDL (Download manager) |
202 | 210 | Pablo Alingery | |
203 | 210 | Pablo Alingery | <pre> |
204 | 210 | Pablo Alingery | apt-get install python3-psycopg2 |
205 | 210 | Pablo Alingery | </pre> |
206 | 210 | Pablo Alingery | |
207 | 210 | Pablo Alingery | Add python3-pip for modules within python3 |
208 | 210 | Pablo Alingery | |
209 | 210 | Pablo Alingery | <pre> |
210 | 210 | Pablo Alingery | apt-get install python3-pip |
211 | 1 | Herve Ballans | </pre> |
212 | 212 | Pablo Alingery | |
213 | 212 | Pablo Alingery | <pre> |
214 | 212 | Pablo Alingery | pip3 install pySmartDL |
215 | 212 | Pablo Alingery | </pre> |
216 | 1 | Herve Ballans | |
217 | 213 | Herve Ballans | Both python2.7 and 3.4 versions are installed on the system. |
218 | 213 | Herve Ballans | By default, the python2.7 is used. |
219 | 213 | Herve Ballans | In order to change in 3.4 by default, type : |
220 | 213 | Herve Ballans | <pre> |
221 | 213 | Herve Ballans | update-alternatives --config python |
222 | 213 | Herve Ballans | </pre> |
223 | 209 | Pablo Alingery | |
224 | 1 | Herve Ballans | h3. Installation of sdo_scripts |
225 | 218 | Pablo Alingery | |
226 | 219 | Pablo Alingery | Install monitoring scripts for sdo |
227 | 218 | Pablo Alingery | |
228 | 218 | Pablo Alingery | <pre> |
229 | 268 | Pablo Alingery | rsync -av scripts production@sdo-new:/home/production/netdrms-tools/ |
230 | 218 | Pablo Alingery | </pre> |
231 | 218 | Pablo Alingery | |
232 | 349 | Herve Ballans | h3. lib64 symbolic link |
233 | 349 | Herve Ballans | |
234 | 349 | Herve Ballans | Creating lib64 link (lib64 does not exist on debian 8) |
235 | 349 | Herve Ballans | <pre> |
236 | 349 | Herve Ballans | # cd /usr |
237 | 349 | Herve Ballans | # ln -s lib/ lib64 |
238 | 349 | Herve Ballans | </pre> |
239 | 349 | Herve Ballans | |
240 | 349 | Herve Ballans | |
241 | 1 | Herve Ballans | h2. NetDRMS databases |
242 | 1 | Herve Ballans | |
243 | 156 | Pablo Alingery | into /var/lib/postgresql (hard mount point separated, 7 disks 15000 tr/min in raid5 in order to distribute read and write) |
244 | 1 | Herve Ballans | |
245 | 1 | Herve Ballans | <pre> |
246 | 158 | Pablo Alingery | # cd /var/lib/postgresql |
247 | 157 | Pablo Alingery | # mkdir data data_sums data_monitor |
248 | 157 | Pablo Alingery | # chown -R postgres:postgres * |
249 | 1 | Herve Ballans | </pre> |
250 | 1 | Herve Ballans | |
251 | 160 | Pablo Alingery | h3. Initialize 3 servers data, data_sums, data_monitor |
252 | 1 | Herve Ballans | |
253 | 159 | Pablo Alingery | Into postgres |
254 | 1 | Herve Ballans | <pre> |
255 | 1 | Herve Ballans | # su - postgres |
256 | 1 | Herve Ballans | |
257 | 1 | Herve Ballans | $ initdb --locale=C -D data -A md5 -W |
258 | 1 | Herve Ballans | (no password) |
259 | 2 | Herve Ballans | |
260 | 2 | Herve Ballans | Success. You can now start the database server using: |
261 | 2 | Herve Ballans | |
262 | 2 | Herve Ballans | postgres -D data |
263 | 2 | Herve Ballans | or |
264 | 2 | Herve Ballans | pg_ctl -D data -l logfile start |
265 | 2 | Herve Ballans | |
266 | 1 | Herve Ballans | |
267 | 1 | Herve Ballans | $ initdb --locale=C -D data_sums -A md5 -W |
268 | 2 | Herve Ballans | (no password) |
269 | 2 | Herve Ballans | |
270 | 2 | Herve Ballans | Success. You can now start the database server using: |
271 | 2 | Herve Ballans | |
272 | 2 | Herve Ballans | postgres -D data_sums |
273 | 2 | Herve Ballans | or |
274 | 2 | Herve Ballans | pg_ctl -D data_sums -l logfile start |
275 | 8 | Pablo Alingery | |
276 | 8 | Pablo Alingery | $ initdb --locale=C -D data_monitor -A md5 -W |
277 | 8 | Pablo Alingery | (no password) |
278 | 8 | Pablo Alingery | |
279 | 8 | Pablo Alingery | Success. You can now start the database server using: |
280 | 8 | Pablo Alingery | |
281 | 8 | Pablo Alingery | postgres -D data_monitor |
282 | 8 | Pablo Alingery | or |
283 | 8 | Pablo Alingery | pg_ctl -D data_monitor -l logfile start |
284 | 2 | Herve Ballans | </pre> |
285 | 2 | Herve Ballans | |
286 | 1 | Herve Ballans | |
287 | 1 | Herve Ballans | |
288 | 1 | Herve Ballans | Edition des fichiers postgresql.conf pour configurer les bons ports d'écoute : |
289 | 1 | Herve Ballans | listen_addresses = '*' |
290 | 1 | Herve Ballans | data -> port 5432 |
291 | 10 | Pablo Alingery | data_sums -> port 5434 |
292 | 1 | Herve Ballans | data_monitor -> port 5436 |
293 | 10 | Pablo Alingery | |
294 | 165 | Pablo Alingery | Edit files data*/pg_hba.conf : replace 'md5' by 'trust' |
295 | 1 | Herve Ballans | <pre> |
296 | 1 | Herve Ballans | #"local" |
297 | 1 | Herve Ballans | local all all trust |
298 | 1 | Herve Ballans | #"IPv4" |
299 | 1 | Herve Ballans | host all all trust |
300 | 1 | Herve Ballans | </pre> |
301 | 166 | Pablo Alingery | |
302 | 166 | Pablo Alingery | h3. Tunning postgres |
303 | 166 | Pablo Alingery | |
304 | 166 | Pablo Alingery | In file postgres.conf ( according to https://www.qwant.com/?q=Tunning+postgres+server+9.4+&client=firefox ) |
305 | 166 | Pablo Alingery | |
306 | 168 | Pablo Alingery | shared_buffer=4096MB %{color:red}So 1/12 of the total RAM memory available% |
307 | 1 | Herve Ballans | |
308 | 169 | Pablo Alingery | work_mem = 24MB |
309 | 168 | Pablo Alingery | |
310 | 168 | Pablo Alingery | maintenance_work_mem = 4096MB |
311 | 1 | Herve Ballans | |
312 | 169 | Pablo Alingery | wal_buffers = 16MB |
313 | 169 | Pablo Alingery | |
314 | 169 | Pablo Alingery | checkpoint_segments = 32 |
315 | 169 | Pablo Alingery | |
316 | 169 | Pablo Alingery | checkpoint_completion_target = 0.9 |
317 | 169 | Pablo Alingery | |
318 | 169 | Pablo Alingery | random_page_cost = 1.0 |
319 | 169 | Pablo Alingery | |
320 | 169 | Pablo Alingery | effective_cache_size = 16384MB so 1/4 of the max memory |
321 | 169 | Pablo Alingery | |
322 | 170 | Pablo Alingery | |
323 | 170 | Pablo Alingery | |
324 | 170 | Pablo Alingery | #log |
325 | 169 | Pablo Alingery | logging_collector = on |
326 | 169 | Pablo Alingery | log_directory = 'pg_log' |
327 | 169 | Pablo Alingery | log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # log file name pattern, |
328 | 169 | Pablo Alingery | log_truncate_on_rotation = on # If on, an existing log file with the |
329 | 169 | Pablo Alingery | log_rotation_age = 1d # Automatic rotation of logfiles will |
330 | 170 | Pablo Alingery | client_min_messages = error |
331 | 170 | Pablo Alingery | log_min_messages = error |
332 | 170 | Pablo Alingery | log_min_duration_statement = 0 |
333 | 170 | Pablo Alingery | |
334 | 170 | Pablo Alingery | log_connections = on |
335 | 170 | Pablo Alingery | log_disconnections = on |
336 | 170 | Pablo Alingery | log_duration = off |
337 | 170 | Pablo Alingery | log_hostname = on |
338 | 170 | Pablo Alingery | log_line_prefix = '%t [%p]: %u@%h - %d :' |
339 | 170 | Pablo Alingery | |
340 | 170 | Pablo Alingery | track_counts = on |
341 | 170 | Pablo Alingery | autovacuum = on |
342 | 170 | Pablo Alingery | |
343 | 170 | Pablo Alingery | extra_float_digits = 3 |
344 | 170 | Pablo Alingery | |
345 | 177 | Pablo Alingery | h3. Start the 3 databases : |
346 | 1 | Herve Ballans | |
347 | 1 | Herve Ballans | <pre> |
348 | 1 | Herve Ballans | # su - postgres |
349 | 196 | Herve Ballans | $ cd /var/lib/postgresql |
350 | 172 | Pablo Alingery | $ pg_ctl -D data start |
351 | 172 | Pablo Alingery | $ pg_ctl -D data_sums start |
352 | 172 | Pablo Alingery | $ pg_ctl -D data_monitor start |
353 | 1 | Herve Ballans | </pre> |
354 | 174 | Pablo Alingery | |
355 | 1 | Herve Ballans | h3. Create the 3 databases : |
356 | 181 | Pablo Alingery | |
357 | 178 | Pablo Alingery | Into user 'postgres' |
358 | 1 | Herve Ballans | |
359 | 179 | Pablo Alingery | * createdb --locale C -E LATIN1 -T template0 ias_sdo |
360 | 179 | Pablo Alingery | * createdb --locale C -E LATIN1 -T template0 -p 5434 ias_sdo_sums |
361 | 179 | Pablo Alingery | * createdb --locale C -E LATIN1 -T template0 -p 5436 ias_sdo_monitor |
362 | 54 | Herve Ballans | |
363 | 182 | Herve Ballans | h3. Automatic startup |
364 | 182 | Herve Ballans | |
365 | 182 | Herve Ballans | Add following lines in the /etc/rc.local file : |
366 | 182 | Herve Ballans | |
367 | 182 | Herve Ballans | <pre> |
368 | 197 | Herve Ballans | su -l -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D /var/lib/postgresql/data start" postgres |
369 | 197 | Herve Ballans | su -l -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D /var/lib/postgresql/data_sums start" postgres |
370 | 197 | Herve Ballans | su -l -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D /var/lib/postgresql/data_monitor start" postgres |
371 | 182 | Herve Ballans | </pre> |
372 | 182 | Herve Ballans | |
373 | 325 | Pablo Alingery | h3. Install additional modules |
374 | 327 | Pablo Alingery | |
375 | 326 | Pablo Alingery | We need dblink for triggers between db so let's install postgresql-contrib-9.4 |
376 | 325 | Pablo Alingery | |
377 | 325 | Pablo Alingery | <pre> |
378 | 326 | Pablo Alingery | root@sdo-new:~# apt-get install postgresql-contrib-9.4 |
379 | 325 | Pablo Alingery | </pre> |
380 | 325 | Pablo Alingery | |
381 | 328 | Pablo Alingery | Create extension dblink for each db |
382 | 328 | Pablo Alingery | ias_sdo |
383 | 325 | Pablo Alingery | <pre> |
384 | 328 | Pablo Alingery | root@sdo-new:$ psql ias_sdo -p5432 -U postgres |
385 | 328 | Pablo Alingery | psql (9.4.6) |
386 | 325 | Pablo Alingery | Type "help" for help. |
387 | 1 | Herve Ballans | |
388 | 328 | Pablo Alingery | ias_sdo=# CREATE EXTENSION dblink ; |
389 | 1 | Herve Ballans | CREATE EXTENSION |
390 | 328 | Pablo Alingery | </pre> |
391 | 1 | Herve Ballans | |
392 | 328 | Pablo Alingery | ias_sdo_sums |
393 | 328 | Pablo Alingery | <pre> |
394 | 328 | Pablo Alingery | root@sdo-new:$ psql ias_sdo -p5434 -U postgres |
395 | 328 | Pablo Alingery | psql (9.4.6) |
396 | 328 | Pablo Alingery | Type "help" for help. |
397 | 328 | Pablo Alingery | |
398 | 328 | Pablo Alingery | ias_sdo=# CREATE EXTENSION dblink ; |
399 | 328 | Pablo Alingery | CREATE EXTENSION |
400 | 328 | Pablo Alingery | </pre> |
401 | 328 | Pablo Alingery | |
402 | 328 | Pablo Alingery | ias_sdo_monitor |
403 | 328 | Pablo Alingery | <pre> |
404 | 328 | Pablo Alingery | root@sdo-new:$ psql ias_sdo -p5436 -U postgres |
405 | 328 | Pablo Alingery | psql (9.4.6) |
406 | 328 | Pablo Alingery | Type "help" for help. |
407 | 328 | Pablo Alingery | |
408 | 328 | Pablo Alingery | ias_sdo=# CREATE EXTENSION dblink ; |
409 | 328 | Pablo Alingery | CREATE EXTENSION |
410 | 325 | Pablo Alingery | </pre> |
411 | 325 | Pablo Alingery | |
412 | 330 | Pablo Alingery | To check the list of installed extensions type : |
413 | 330 | Pablo Alingery | <pre> |
414 | 330 | Pablo Alingery | ias_sdo=\dx |
415 | 332 | Herve Ballans | List of installed extensions |
416 | 332 | Herve Ballans | Name | Version | Schema | Description |
417 | 332 | Herve Ballans | ---------+---------+------------+-------------------------------------------------------------- |
418 | 332 | Herve Ballans | dblink | 1.1 | public | connect to other PostgreSQL databases from within a database |
419 | 332 | Herve Ballans | plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language |
420 | 332 | Herve Ballans | (2 rows) |
421 | 330 | Pablo Alingery | |
422 | 330 | Pablo Alingery | </pre> |
423 | 332 | Herve Ballans | |
424 | 332 | Herve Ballans | |
425 | 325 | Pablo Alingery | |
426 | 184 | Herve Ballans | h2. System tuning |
427 | 1 | Herve Ballans | |
428 | 187 | Herve Ballans | Current configuration : |
429 | 185 | Herve Ballans | <pre> |
430 | 185 | Herve Ballans | # sysctl -a |
431 | 185 | Herve Ballans | </pre> |
432 | 1 | Herve Ballans | |
433 | 187 | Herve Ballans | In the following file : |
434 | 185 | Herve Ballans | <pre> |
435 | 185 | Herve Ballans | # vi /etc/sysctl.conf |
436 | 185 | Herve Ballans | </pre> |
437 | 1 | Herve Ballans | |
438 | 188 | Herve Ballans | we change some values (default values are commented with '#') |
439 | 194 | Herve Ballans | |
440 | 195 | Herve Ballans | (resources links [[Links|here]]) |
441 | 194 | Herve Ballans | |
442 | 185 | Herve Ballans | <pre> |
443 | 191 | Herve Ballans | # Semaphore is a object that is used to control utilization of a particular process. |
444 | 191 | Herve Ballans | # kernel.sem = SEMMSL SEMMNS SEMOPM SEMMNI |
445 | 191 | Herve Ballans | # SEMMSL maximum number of semaphores per array (min 128) |
446 | 191 | Herve Ballans | # SEMMNS maximum semaphores system-wide |
447 | 191 | Herve Ballans | # SEMOPM maximum operations per semop call |
448 | 191 | Herve Ballans | # SEMMNI maximum arrays |
449 | 191 | Herve Ballans | #semop incresed from 32 (default value)operation to 100 (3rd value ) |
450 | 191 | Herve Ballans | # kernel.sem = 250 32000 32 128 |
451 | 191 | Herve Ballans | kernel.sem = 250 32000 100 128 |
452 | 191 | Herve Ballans | |
453 | 191 | Herve Ballans | In order to apply changes :#This sets the OS receive buffer size for all types of connections (default value 212992) |
454 | 191 | Herve Ballans | #net.core.rmem_default = 212992 |
455 | 191 | Herve Ballans | #net.core.rmem_max = 212992 |
456 | 191 | Herve Ballans | net.core.rmem_default = 33554432 |
457 | 191 | Herve Ballans | net.core.rmem_max = 33554432 |
458 | 191 | Herve Ballans | |
459 | 191 | Herve Ballans | #This is the OS send buffer size for all types of connections (212992 default value) |
460 | 191 | Herve Ballans | #net.core.wmem_default = 212992 |
461 | 191 | Herve Ballans | #net.core.wmem_max = 212992 |
462 | 191 | Herve Ballans | net.core.wmem_default = 33554432 |
463 | 191 | Herve Ballans | net.core.wmem_max = 33554432 |
464 | 191 | Herve Ballans | |
465 | 193 | Pablo Alingery | #TCP Autotuning setting. "The first value tells the kernel the minimum receive buffer for each TCP connection, and this buffer is always |
466 | 193 | Pablo Alingery | # allocated to a TCP socket, even under high pressure on the system. ... The second value specified tells the kernel the default receive |
467 | 193 | Pablo Alingery | # buffer allocated for each TCP socket. This value overrides the /proc/sys/net/core/rmem_default value used by other protocols. ... |
468 | 192 | Pablo Alingery | # The third and last value specified in this variable specifies the maximum receive buffer that can be allocated for a TCP socket." |
469 | 1 | Herve Ballans | #net.ipv4.tcp_rmem = 4096 87380 6291456 |
470 | 1 | Herve Ballans | net.ipv4.tcp_rmem = 10240 87380 33554432 |
471 | 1 | Herve Ballans | |
472 | 193 | Pablo Alingery | #TCP Autotuning setting. "This variable takes 3 different values which holds information on how much TCP sendbuffer memory space |
473 | 193 | Pablo Alingery | # each TCP socket has to use. Every TCP socket has this much buffer space to use before the buffer is filled up. Each of the three |
474 | 193 | Pablo Alingery | #values are used under different conditions. ... The first value in this variable tells the minimum TCP send buffer space available |
475 | 193 | Pablo Alingery | # for a single TCP socket. ... The second value in the variable tells us the default buffer space allowed for a single TCP socket to use. ... |
476 | 193 | Pablo Alingery | #The third value tells the kernel the maximum TCP send buffer space." |
477 | 191 | Herve Ballans | #net.ipv4.tcp_wmem = 4096 16384 4194304 |
478 | 191 | Herve Ballans | net.ipv4.tcp_wmem = 10240 87380 33554432 |
479 | 191 | Herve Ballans | |
480 | 191 | Herve Ballans | #Disable cache metrics so the initial conditions of the closed connections will not be saved to be used in near future connections |
481 | 191 | Herve Ballans | #net.ipv4.tcp_no_metrics_save = 0 |
482 | 191 | Herve Ballans | net.ipv4.tcp_no_metrics_save = 1 |
483 | 191 | Herve Ballans | |
484 | 1 | Herve Ballans | # Increase number of incoming connections backlog |
485 | 1 | Herve Ballans | # net.core.netdev_max_backlog = 1000 |
486 | 1 | Herve Ballans | net.core.netdev_max_backlog = 5000 |
487 | 1 | Herve Ballans | |
488 | 1 | Herve Ballans | # The tcp_mem variable defines how the TCP stack should behave when it comes to memory usage. ... |
489 | 192 | Pablo Alingery | #The first value specified in the tcp_mem variable tells the kernel the low threshold. |
490 | 192 | Pablo Alingery | #Below this point, the TCP stack do not bother at all about putting any pressure on the memory usage by different TCP sockets. ... |
491 | 193 | Pablo Alingery | #The second value tells the kernel at which point to start pressuring memory usage down. ... The final value tells the kernel how many |
492 | 193 | Pablo Alingery | # memory pages it may use maximally |
493 | 1 | Herve Ballans | # net.ipv4.tcp_mem = 1543347 2057796 3086694 |
494 | 191 | Herve Ballans | net.ipv4.tcp_mem = 786432 1048576 26777216 |
495 | 191 | Herve Ballans | |
496 | 192 | Pablo Alingery | # local port range that is used by TCP and UDP traffic to choose the local port. You will see in the parameters of this file two numbers: |
497 | 192 | Pablo Alingery | # The first number is the first local port allowed for TCP and UDP traffic on the server, the second is the last local port number. |
498 | 191 | Herve Ballans | # net.ipv4.ip_local_port_range = 32768 61000 |
499 | 191 | Herve Ballans | net.ipv4.ip_local_port_range = 1024 65535 |
500 | 191 | Herve Ballans | |
501 | 191 | Herve Ballans | # maximum number of sockets in TIME-WAIT to be held simultaneously. |
502 | 191 | Herve Ballans | # net.ipv4.tcp_max_tw_buckets = 262144 |
503 | 191 | Herve Ballans | net.ipv4.tcp_max_tw_buckets = 360000 |
504 | 1 | Herve Ballans | </pre> |
505 | 185 | Herve Ballans | |
506 | 191 | Herve Ballans | Then we run the following command to make our change take effect: |
507 | 191 | Herve Ballans | |
508 | 185 | Herve Ballans | <pre> |
509 | 185 | Herve Ballans | # sysctl -p |
510 | 185 | Herve Ballans | </pre> |
511 | 176 | Pablo Alingery | |
512 | 216 | Pablo Alingery | h2. JMD Installation |
513 | 215 | Herve Ballans | |
514 | 217 | Pablo Alingery | Download new JMD package build by Niles Oien 2016-04-07 following thoses actions : |
515 | 215 | Herve Ballans | |
516 | 215 | Herve Ballans | <pre> |
517 | 215 | Herve Ballans | cd /urs/local |
518 | 215 | Herve Ballans | |
519 | 215 | Herve Ballans | [oien@spsc-nso19-12 ~]$ ftp gong2.nso.edu |
520 | 215 | Herve Ballans | Name (gong2.nso.edu:oien): anonymous |
521 | 215 | Herve Ballans | Password: <---------------- use your email as the password |
522 | 215 | Herve Ballans | ftp> cd outgoing/oien |
523 | 215 | Herve Ballans | ftp> get pablo_jmd.tar.gz |
524 | 215 | Herve Ballans | ftp> quit |
525 | 215 | Herve Ballans | </pre> |
526 | 215 | Herve Ballans | |
527 | 215 | Herve Ballans | Install new JMD package following thoses actions |
528 | 215 | Herve Ballans | |
529 | 215 | Herve Ballans | <pre> |
530 | 215 | Herve Ballans | [oien@spsc-nso19-12 ~]$ gunzip -vf pablo_jmd.tar.gz |
531 | 215 | Herve Ballans | pablo_jmd.tar.gz: 8.3% -- replaced with pablo_jmd.tar |
532 | 215 | Herve Ballans | [oien@spsc-nso19-12 ~]$ tar xf pablo_jmd.tar |
533 | 215 | Herve Ballans | [oien@spsc-nso19-12 ~]$ cd pablo_jmd |
534 | 338 | Pablo Alingery | [oien@spsc-nso19-12 pablo_jmd]$ less README.txt |
535 | 215 | Herve Ballans | cd .. |
536 | 215 | Herve Ballans | mv jmd jmd_old |
537 | 215 | Herve Ballans | mv pablo_jmd jmd |
538 | 215 | Herve Ballans | </pre> |
539 | 215 | Herve Ballans | |
540 | 340 | Pablo Alingery | url provided by jsoc to download last JMD version http://vso.tuc.noao.edu/VSO/downloads/JMD |
541 | 339 | Pablo Alingery | |
542 | 266 | Herve Ballans | h2. Apache Configuration for JMD |
543 | 215 | Herve Ballans | |
544 | 215 | Herve Ballans | Configure webserver to request jsoc_fetch |
545 | 215 | Herve Ballans | (following the mail from nilesoien@gmail.com) |
546 | 215 | Herve Ballans | |
547 | 222 | Pablo Alingery | %{color:red}WARNING : change hostname when migrate sdo-new to sdo% |
548 | 222 | Pablo Alingery | |
549 | 220 | Pablo Alingery | config d 'apache2 fichier sdo-new.ias.u-psud.fr |
550 | 215 | Herve Ballans | <pre> |
551 | 215 | Herve Ballans | |
552 | 215 | Herve Ballans | <VirtualHost *:80> |
553 | 220 | Pablo Alingery | ServerName sdo-new.ias.u-psud.fr |
554 | 215 | Herve Ballans | ServerAdmin pablo.alingery@ias.u-psud.fr |
555 | 215 | Herve Ballans | #ErrorLog /var/log/apache2/error.log |
556 | 215 | Herve Ballans | DocumentRoot /var/www |
557 | 215 | Herve Ballans | |
558 | 215 | Herve Ballans | # Possible values include: debug, info, notice, warn, error, crit, |
559 | 215 | Herve Ballans | # alert, emerg. |
560 | 215 | Herve Ballans | LogLevel debug |
561 | 215 | Herve Ballans | |
562 | 215 | Herve Ballans | #CustomLog /var/log/apache2/VSO/access.log combined |
563 | 215 | Herve Ballans | |
564 | 215 | Herve Ballans | Alias /VSO/DRMS/cgi-bin/ "/home/production/netdrms-tools/scripts/cgi-bin/" |
565 | 215 | Herve Ballans | |
566 | 215 | Herve Ballans | <Location /VSO/DRMS/cgi-bin> |
567 | 215 | Herve Ballans | SetHandler perl-script |
568 | 215 | Herve Ballans | PerlResponseHandler ModPerl::Registry |
569 | 215 | Herve Ballans | PerlOptions +ParseHeaders |
570 | 215 | Herve Ballans | Options +ExecCGI |
571 | 215 | Herve Ballans | Order allow,deny |
572 | 215 | Herve Ballans | Allow from all |
573 | 215 | Herve Ballans | </Location> |
574 | 215 | Herve Ballans | </VirtualHost> |
575 | 215 | Herve Ballans | |
576 | 215 | Herve Ballans | </pre> |
577 | 215 | Herve Ballans | |
578 | 223 | Herve Ballans | Activate the new virtualhost and disable the default one : |
579 | 223 | Herve Ballans | <pre> |
580 | 225 | Herve Ballans | # a2ensite sdo-new.ias.u-psud.fr.conf |
581 | 225 | Herve Ballans | # a2dissite 000-default.conf |
582 | 225 | Herve Ballans | # service apache2 restart |
583 | 225 | Herve Ballans | </pre> |
584 | 225 | Herve Ballans | |
585 | 225 | Herve Ballans | Allow symlink for the cgi scripts dierctory in apache2.conf : |
586 | 225 | Herve Ballans | <pre> |
587 | 225 | Herve Ballans | <Directory /home/production/netdrms-tools/scripts/> |
588 | 225 | Herve Ballans | Options FollowSymLinks |
589 | 225 | Herve Ballans | AllowOverride None |
590 | 225 | Herve Ballans | Require all granted |
591 | 225 | Herve Ballans | </Directory> |
592 | 225 | Herve Ballans | </pre> |
593 | 225 | Herve Ballans | |
594 | 225 | Herve Ballans | <pre> |
595 | 225 | Herve Ballans | # service apache2 restart |
596 | 223 | Herve Ballans | </pre> |
597 | 223 | Herve Ballans | |
598 | 226 | Pablo Alingery | %{color:red}To be added futher to test cgi% |
599 | 1 | Herve Ballans | Execution test cgi with URL : |
600 | 226 | Pablo Alingery | |
601 | 226 | Pablo Alingery | make sure my $hostname=in the cgi file vso_jsoc_fetch.cgi is correct and test |
602 | 224 | Pablo Alingery | http://sdo-new.ias.u-psud.fr/VSO/DRMS/cgi-bin/vso_jsoc_fetch.cgi |
603 | 215 | Herve Ballans | |
604 | 215 | Herve Ballans | expected result : |
605 | 215 | Herve Ballans | <pre> |
606 | 215 | Herve Ballans | { |
607 | 215 | Herve Ballans | "wait" : 0, |
608 | 215 | Herve Ballans | "requestid" : "", |
609 | 215 | Herve Ballans | "data" : {}, |
610 | 215 | Herve Ballans | "method" : "url_quick", |
611 | 215 | Herve Ballans | "size" : 0, |
612 | 215 | Herve Ballans | "errormsg" : "Empty query", |
613 | 215 | Herve Ballans | "protocol" : "as-is", |
614 | 215 | Herve Ballans | "status" : 1, |
615 | 215 | Herve Ballans | "space_ratio" : 0.1, |
616 | 215 | Herve Ballans | "load_ratio" : 0.066875, |
617 | 215 | Herve Ballans | "dir" : "", |
618 | 215 | Herve Ballans | "count" : 0 |
619 | 215 | Herve Ballans | } |
620 | 215 | Herve Ballans | </pre> |
621 | 215 | Herve Ballans | |
622 | 267 | Herve Ballans | h2. JMD configuration |
623 | 267 | Herve Ballans | |
624 | 291 | Herve Ballans | Modify the file [[/usr/local/jmd/cfg/JMD.cfg]] |
625 | 267 | Herve Ballans | |
626 | 267 | Herve Ballans | |
627 | 271 | Herve Ballans | %{color:red}Warning : when migrate production server, change _sdo-new_ by _sdo_% |
628 | 271 | Herve Ballans | |
629 | 1 | Herve Ballans | h2. NetDRMS tool Install |
630 | 23 | Pablo Alingery | |
631 | 199 | Pablo Alingery | The following installed is based on : [[http://inf-redmine.ias.u-psud.fr/redmine/attachments/download/114/Netdrms_First_Install.odt]] |
632 | 1 | Herve Ballans | |
633 | 207 | Pablo Alingery | The sources are in : http://jsoc.stanford.edu/netdrms/dist/ |
634 | 207 | Pablo Alingery | Locally on sdo-new in : /home/production/sources |
635 | 1 | Herve Ballans | |
636 | 199 | Pablo Alingery | Symbolic link created 'netdrms_current' that corresponds to the last release |
637 | 350 | Pablo Alingery | <pre> |
638 | 350 | Pablo Alingery | ln -s netdrms_8.12/ netdrms_current |
639 | 350 | Pablo Alingery | </pre> |
640 | 360 | Pablo Alingery | Edit file config.local and make sure that resource exist |
641 | 360 | Pablo Alingery | |
642 | 360 | Pablo Alingery | for example : |
643 | 360 | Pablo Alingery | <pre> |
644 | 360 | Pablo Alingery | # defaults |
645 | 360 | Pablo Alingery | # PostgreSQL API headers (must contain libpq-fe.h) |
646 | 360 | Pablo Alingery | POSTGRES_INCS:X86_64 /usr/include/postgresql |
647 | 360 | Pablo Alingery | POSTGRES_INCS:AVX /usr/include/postgresql |
648 | 360 | Pablo Alingery | |
649 | 360 | Pablo Alingery | # the location of the PostgreSQL libs; likely to be either |
650 | 360 | Pablo Alingery | # /usr/lib or /usr/lib64 or /usr/local/pgsql/lib |
651 | 360 | Pablo Alingery | POSTGRES_LIBS:X86_64 /usr/lib/postgresql/9.4/lib |
652 | 360 | Pablo Alingery | POSTGRES_LIBS:AVX /usr/lib/postgresql/9.4/lib |
653 | 360 | Pablo Alingery | </pre> |
654 | 360 | Pablo Alingery | |
655 | 360 | Pablo Alingery | So make sure /usr/include/postgresql does exist and is the location of libpq-fe.h for example (see in config.local comments about POSTGRES API headers ) |
656 | 360 | Pablo Alingery | |
657 | 27 | Pablo Alingery | <pre> |
658 | 1 | Herve Ballans | cp config.local.template config.local |
659 | 1 | Herve Ballans | </pre> |
660 | 29 | Pablo Alingery | |
661 | 350 | Pablo Alingery | %{color:red}Warning WEB_DBUSER apache we think that it should value www maybe , to be checked testing the cgi% |
662 | 1 | Herve Ballans | %{color:red}So check the cgi to figure out which user is used to connect to bd% |
663 | 32 | Herve Ballans | |
664 | 199 | Pablo Alingery | Create 2 followings dirs : |
665 | 1 | Herve Ballans | <pre> |
666 | 350 | Pablo Alingery | mkdir /usr/local/netdrms_8.12/logs/SUM -p |
667 | 350 | Pablo Alingery | mkdir /usr/local/netdrms_8.12/logs/slony -p |
668 | 27 | Pablo Alingery | </pre> |
669 | 1 | Herve Ballans | |
670 | 205 | Pablo Alingery | %{color:red}Warning config.local from line 150 =>222 Remote config to provide data, series to externals% |
671 | 205 | Pablo Alingery | %{color:red}To be checked for PLATO% |
672 | 1 | Herve Ballans | |
673 | 1 | Herve Ballans | Création d'un alias netdrms dans /root/.bashrc |
674 | 27 | Pablo Alingery | <pre> |
675 | 1 | Herve Ballans | alias netdrms='cd /usr/local/netdrms_current' |
676 | 1 | Herve Ballans | </pre> |
677 | 31 | Pablo Alingery | |
678 | 1 | Herve Ballans | h3. Prepare compiling |
679 | 1 | Herve Ballans | |
680 | 1 | Herve Ballans | Into _production_ profile |
681 | 352 | Pablo Alingery | Replace all " $? !=0 " by " $status !=0 " in file configure then tape |
682 | 246 | Herve Ballans | <pre> |
683 | 252 | Herve Ballans | $ ./configure |
684 | 246 | Herve Ballans | </pre> |
685 | 246 | Herve Ballans | |
686 | 364 | Pablo Alingery | Again in netdrms_9.0 |
687 | 347 | Pablo Alingery | |
688 | 358 | Pablo Alingery | Modification du fichier /usr/local/netdrms_current/build/jsoc_machine.csh (Same issue in NetDrms 8.11 ) |
689 | 347 | Pablo Alingery | |
690 | 347 | Pablo Alingery | ligne 28 : |
691 | 347 | Pablo Alingery | |
692 | 347 | Pablo Alingery | case "x86_64": |
693 | 347 | Pablo Alingery | echo linux_x86_64 |
694 | 347 | Pablo Alingery | breaksw |
695 | 364 | Pablo Alingery | |
696 | 364 | Pablo Alingery | Again in netdrms_9.0 |
697 | 347 | Pablo Alingery | |
698 | 347 | Pablo Alingery | Overwriting the existing files in the right place (base/sums/libs/pg) & modif in jsoc_machine.csh |
699 | 347 | Pablo Alingery | => the command 'make' works well. |
700 | 347 | Pablo Alingery | |
701 | 113 | Pablo Alingery | h3. Compilation |
702 | 113 | Pablo Alingery | |
703 | 361 | Pablo Alingery | In file make_basic.mk line 223 |
704 | 359 | Pablo Alingery | look for "GCC_LF_ALL =" |
705 | 361 | Pablo Alingery | add "-lcrypto" |
706 | 353 | Pablo Alingery | |
707 | 353 | Pablo Alingery | |
708 | 249 | Herve Ballans | <pre> |
709 | 253 | Herve Ballans | $ make |
710 | 253 | Herve Ballans | $ make sums |
711 | 113 | Pablo Alingery | </pre> |
712 | 113 | Pablo Alingery | |
713 | 255 | Herve Ballans | h2. SSH-HPN install |
714 | 255 | Herve Ballans | |
715 | 258 | Herve Ballans | Intallation de openssh 7.2p2 car le dernier patch hpn dispo est 7.2hpn14.V10 |
716 | 255 | Herve Ballans | |
717 | 258 | Herve Ballans | web site for open ssh : http://ftp2.fr.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-7.2p2.tar.gz |
718 | 258 | Herve Ballans | patch hpn : https://sourceforge.net/projects/hpnssh/files/HPN-SSH%2014v10%207.2p2/openssh-7_2_P2-hpn-14.10.diff |
719 | 255 | Herve Ballans | |
720 | 256 | Pablo Alingery | In the dir /home/production/sources into 'root' |
721 | 255 | Herve Ballans | |
722 | 255 | Herve Ballans | <pre> |
723 | 258 | Herve Ballans | # tar -xzvf openssh-7.2p2.tar.gz |
724 | 258 | Herve Ballans | # cd openssh-7.2p2/ |
725 | 258 | Herve Ballans | # cat ../openssh-7_2_P2-hpn-14.10.diff |patch -p1 |
726 | 255 | Herve Ballans | # ./configure --prefix=/usr/local/hpn-ssh --with-pam --with-md5-passwords --without-zlib-version-check --with-tcp-wrappers |
727 | 255 | Herve Ballans | # make |
728 | 255 | Herve Ballans | # make install |
729 | 255 | Herve Ballans | </pre> |
730 | 255 | Herve Ballans | |
731 | 255 | Herve Ballans | Depuis sdo : |
732 | 255 | Herve Ballans | <pre> |
733 | 255 | Herve Ballans | # cd /usr/local/hpn-ssh/etc/ |
734 | 259 | Herve Ballans | # rsync -av *key* production@sdo-new:/home/production/ |
735 | 255 | Herve Ballans | </pre> |
736 | 255 | Herve Ballans | |
737 | 273 | Herve Ballans | Following the link : http://vso.stanford.edu/netdrms/rmtsums.html |
738 | 1 | Herve Ballans | <pre> |
739 | 274 | Herve Ballans | $ cd /home/production/ |
740 | 273 | Herve Ballans | $ ssh-agent -c > ~/.ssh-agent_rs |
741 | 285 | Pablo Alingery | </pre> |
742 | 363 | Pablo Alingery | NB : correct file ~/.ssh-agent_rs replace 'setenv' by 'export' and add '=' between var and value |
743 | 285 | Pablo Alingery | <pre> |
744 | 285 | Pablo Alingery | #!/bin/csh |
745 | 285 | Pablo Alingery | export SSH_AUTH_SOCK=/tmp/ssh-9POrTXobhLR4/agent.74272; |
746 | 285 | Pablo Alingery | export SSH_AGENT_PID=74273; |
747 | 285 | Pablo Alingery | echo Agent pid 74273; |
748 | 285 | Pablo Alingery | </pre> |
749 | 285 | Pablo Alingery | |
750 | 285 | Pablo Alingery | <pre> |
751 | 273 | Herve Ballans | $ source ~/.ssh-agent_rs |
752 | 273 | Herve Ballans | $ ssh-add ~/.ssh/id_rsa |
753 | 269 | Herve Ballans | </pre> |
754 | 269 | Herve Ballans | |
755 | 259 | Herve Ballans | Sur sdo-new : |
756 | 255 | Herve Ballans | <pre> |
757 | 260 | Herve Ballans | # mv /home/production/*key* /usr/local/hpn-ssh/etc/ |
758 | 255 | Herve Ballans | </pre> |
759 | 255 | Herve Ballans | |
760 | 255 | Herve Ballans | We have to configure the port number to 55000 |
761 | 255 | Herve Ballans | <pre> |
762 | 255 | Herve Ballans | # vi ssh_config |
763 | 255 | Herve Ballans | </pre> |
764 | 255 | Herve Ballans | |
765 | 255 | Herve Ballans | and add : |
766 | 255 | Herve Ballans | <pre> |
767 | 255 | Herve Ballans | #Port 22 |
768 | 255 | Herve Ballans | Port 55000 |
769 | 255 | Herve Ballans | </pre> |
770 | 255 | Herve Ballans | |
771 | 255 | Herve Ballans | We configure also the server hpn (even we don't use it yet) |
772 | 255 | Herve Ballans | <pre> |
773 | 255 | Herve Ballans | # vi sshd_config |
774 | 255 | Herve Ballans | </pre> |
775 | 255 | Herve Ballans | |
776 | 255 | Herve Ballans | and add : |
777 | 255 | Herve Ballans | <pre> |
778 | 255 | Herve Ballans | #Port 22 |
779 | 255 | Herve Ballans | Port 55000 |
780 | 255 | Herve Ballans | |
781 | 255 | Herve Ballans | #PidFile /var/run/sshd.pid |
782 | 255 | Herve Ballans | PidFile /var/run/sshd.55000.pid |
783 | 255 | Herve Ballans | |
784 | 255 | Herve Ballans | # allow the use of the none cipher |
785 | 255 | Herve Ballans | #NoneEnabled no |
786 | 255 | Herve Ballans | NoneEnabled yes |
787 | 255 | Herve Ballans | </pre> |
788 | 255 | Herve Ballans | |
789 | 265 | Herve Ballans | In order to force the use of SSH-hpn by JMD : |
790 | 265 | Herve Ballans | |
791 | 265 | Herve Ballans | <pre> |
792 | 265 | Herve Ballans | $ vi /usr/local/jmd/cfg/JMD.cfg |
793 | 265 | Herve Ballans | </pre> |
794 | 265 | Herve Ballans | |
795 | 265 | Herve Ballans | and replace the path of scp binary : |
796 | 265 | Herve Ballans | |
797 | 265 | Herve Ballans | <pre> |
798 | 265 | Herve Ballans | SCPCommand=/usr/local/hpn-ssh/bin/scp -o NoneSwitch=yes -o NoneEnabled=yes |
799 | 265 | Herve Ballans | </pre> |
800 | 265 | Herve Ballans | |
801 | 262 | Herve Ballans | h2. Private key Public key generation |
802 | 262 | Herve Ballans | |
803 | 262 | Herve Ballans | JSOC need your public key to directly connect to your netdrms server. In order to generate your RSA public/private key pair : |
804 | 262 | Herve Ballans | |
805 | 262 | Herve Ballans | <pre> |
806 | 262 | Herve Ballans | $ ssh-keygen -t rsa |
807 | 262 | Herve Ballans | </pre> |
808 | 262 | Herve Ballans | |
809 | 263 | Pablo Alingery | Not necessary in our case cause we recover the ssh keys of the current server 'sdo'. |
810 | 262 | Herve Ballans | |
811 | 262 | Herve Ballans | However, we have to retrieve the private key of production account on sdo-new |
812 | 262 | Herve Ballans | |
813 | 262 | Herve Ballans | From sdo : |
814 | 262 | Herve Ballans | <pre> |
815 | 262 | Herve Ballans | rsync -av id_rsa production@sdo-new:/home/production/.ssh/ |
816 | 262 | Herve Ballans | </pre> |
817 | 262 | Herve Ballans | |
818 | 300 | Herve Ballans | In order to test the connection : |
819 | 300 | Herve Ballans | <pre> |
820 | 300 | Herve Ballans | $ ssh jsocexp@jsocport.stanford.edu -p55000 |
821 | 300 | Herve Ballans | </pre> |
822 | 300 | Herve Ballans | |
823 | 301 | Herve Ballans | Note : the IP of the server has to be declared to the jsoc |
824 | 301 | Herve Ballans | |
825 | 299 | Pablo Alingery | h2. NetDRMS Replication Config |
826 | 112 | Herve Ballans | |
827 | 319 | Herve Ballans | %{color:red}To be check if we keep that here or not% |
828 | 311 | Pablo Alingery | To prevent from deleting files |
829 | 307 | Pablo Alingery | <pre> |
830 | 307 | Pablo Alingery | cd logs/SUM ; scp production@sdo:/usr/local/netdrms/install-config-files/sum_rm.cfg . |
831 | 112 | Herve Ballans | </pre> |
832 | 1 | Herve Ballans | |
833 | 1 | Herve Ballans | Create tmp dir in netdrms working directory (/usr/local/netdrms_current) |
834 | 112 | Herve Ballans | <pre> |
835 | 315 | Herve Ballans | $ mkdir tmp |
836 | 112 | Herve Ballans | </pre> |
837 | 112 | Herve Ballans | |
838 | 308 | Pablo Alingery | Create config file from template |
839 | 112 | Herve Ballans | <pre> |
840 | 315 | Herve Ballans | $ cd base/drms/replication/etc |
841 | 313 | Pablo Alingery | cp repclient.template.cfg ias.repclient.cfg |
842 | 307 | Pablo Alingery | </pre> |
843 | 91 | Herve Ballans | |
844 | 316 | Herve Ballans | Define node in file : [[/usr/local/netdrms_current/base/drms/replication/etc/ias.repclient.cfg]] |
845 | 1 | Herve Ballans | |
846 | 309 | Pablo Alingery | In our case : |
847 | 94 | Herve Ballans | <pre> |
848 | 1 | Herve Ballans | node=IASprod |
849 | 319 | Herve Ballans | </pre> |
850 | 319 | Herve Ballans | |
851 | 341 | Pablo Alingery | There are 4 new fields in the new version of repclient (see Art mail June 11, 2016) |
852 | 335 | Herve Ballans | |
853 | 319 | Herve Ballans | <pre> |
854 | 320 | Herve Ballans | #kRSBaseURL=<base URL for all Slony services> |
855 | 334 | Herve Ballans | kRSBaseURL=http://jsoc.stanford.edu/cgi-bin/ajax |
856 | 320 | Herve Ballans | #kSubService=<URL of the subscription service> |
857 | 334 | Herve Ballans | kSubService=${kRSBaseURL}/request-subs.py |
858 | 320 | Herve Ballans | #kPubListService=<URL of the publication-list service> |
859 | 334 | Herve Ballans | kPubListService=${kRSBaseURL}/publist.py |
860 | 320 | Herve Ballans | #kSubXfer=<URL of the file transfer directory> |
861 | 334 | Herve Ballans | kSubXfer=http://jsoc.stanford.edu/subscription |
862 | 106 | Pablo Alingery | </pre> |
863 | 321 | Pablo Alingery | |
864 | 321 | Pablo Alingery | h2. NetDRMS Subsribe list Config |
865 | 321 | Pablo Alingery | |
866 | 321 | Pablo Alingery | Create config subscribe file from template |
867 | 321 | Pablo Alingery | <pre> |
868 | 321 | Pablo Alingery | $ cd base/drms/replication/etc |
869 | 321 | Pablo Alingery | cp subscribe_list.template.cfg ias.subscribe_list.cfg |
870 | 321 | Pablo Alingery | </pre> |
871 | 321 | Pablo Alingery | |
872 | 292 | Pablo Alingery | h2. TO BE DONE BEFORE MIGRATION |
873 | 1 | Herve Ballans | |
874 | 152 | Pablo Alingery | %{color:red} configure sytem taille buffer% |
875 | 152 | Pablo Alingery | |
876 | 155 | Pablo Alingery | %{color:red} swap désactivé ? discussion Stephane et Gilles% |
877 | 1 | Herve Ballans | |
878 | 1 | Herve Ballans | %{color:red}Test ssh connection to JSOC , NSO CFA before subcription and just after name sdo-new into sdo% |
879 | 317 | Pablo Alingery | %{color:red}Take old counter and place it in the same dir before migration% |
880 | 354 | Pablo Alingery | |
881 | 354 | Pablo Alingery | %{color:red}Before import existing db into postgres dont't forget to drop old one% |
882 | 356 | Pablo Alingery | |
883 | 357 | Pablo Alingery | %{color:red}Install VSO, test cgi to access hmi/aia data% |