CRIN Trac: Ticket #23: Development and staging environment

hours changed; totalhours set

chris — Wed, 17 Jun 2015 16:37:34 GMT

hours changed from 0 to 0.15
totalhours set to 0.15

Replying to peter:

We would like to set up an environment that is as close as possible to the production environment as possible, so that we are able to trial and review any changes to the site before they go live.

That makes sense.

We would ideally have two environments dev and stage on the same box.

That shouldn't be a problem.

Ideally we would use dev.crin.org and stage.crin.org as two document roots

Makes sense.

We will script any deployment from git

Good idea.

the same script should be used on productin too

Makse sense.

We will use drush to sync between production and staging/dev

There is an issue with drush I would need to resolve first, it doesn't support SSL/TLS MySQL connections and currently the Crin2 web server connects using SSL/TLS MySQL to Crin1 so I think we would need to change that to use stunnel or ssh, see ticket:18.

Also today I have set the Zend OPcache to not check for updated PHP files (the php files are all owned by root currently and there is no way for them to be updated!), see ticket:9#comment:16 so a service php5-fpm restart or something would be needed after each deplyment of updated PHP files, but that shouldn't be a problem.

we will need to have ssh access between the stage box and the prod box to be able to sync

No problem.

I would also like to know how you manage updates on the boxes. In the long run it is pretty important that the dev box is kept in sync with the prod box.

I have been applying Debian security updates as they are available using this script, wiki:AptitudeUpdateScript which records the updates in /root/Changelog and then the Changelog is emailed by logwatch and I have also been recording the updates on ticket:17, this is all quite quick to do and I could easilly do it for a dev box.

hours, totalhours changed

chris — Thu, 18 Jun 2015 09:02:11 GMT

hours changed from 0 to 0.15
totalhours changed from 0.15 to 0.3

I have a few more thoughts on this, how do these suggestions sound?

Using the same database server for the dev and live servers would be a cost saving.
The live server has 4GB of RAM, a dev server might be able to get away with 512MB or 1GB, this would make it slower but cheaper.
The key differences that the live and dev/stage sites need to have are:
- A robots.txt to ensure that the site isn't indexed be search engines.
- One or more methods (eg a database edit or a MTA setting or Drupal module) to ensure that the dev/stage sites never send emails to anyone other than developers.
A script to sync the files and database from live to dev/stage which potentially also omits cache and log tables and also potentially edits all email addresses, this could be on the database server to make it quicker to run.

hours, totalhours changed

chris — Thu, 18 Jun 2015 09:10:37 GMT

hours changed from 0 to 0.05
totalhours changed from 0.3 to 0.35

One further thought, the git repository could potentially be hosted on the same server as Trac (Crin1) and it could be integrated with Trac so that the code could be browsed via Trac etc, see http://trac.edgewall.org/wiki/TracGit

peter — Fri, 26 Jun 2015 16:40:04 GMT

Hi Chris here is my current technical writeup.
= Development server =
------------------------------
== Domains
* dev.crin.org
* stage.crin.org
== Resources
1GB Ram
Both sites will be hosted on the same server.
Database server shared with production.
= Repository
All code will be managed via a source repository "owned" by the client.
= Functionality
== Deployment scripts
In order to ensure consistency, all deployment should take place in a
consistent manner.
== Database and file sync
Allow developers to safely sync between environments
prod -> stage -> dev -> local
== Database snapshot and restore
Drush based database snapshot with reliable, tested, roll-back.
= Tasks
*deployment scripts
  * automatic db snapshot before deployment (on prod environment)
  * dedicated deployment user
    *developers should be able to sudo to deploy user
* apache environment setup
  * Allow scripts and code to be aware of environment
  * correct file ownership permissions for webserver and deployment user
    * umask and group ownership takes some fiddling
* sync scripts
  * drush based sync
  * ensure that dev/stage sites have shield module enabled to disable
indexing
* drush setup
   * aliases
      *  prod, stage,dev, local
      *  part of code repository to allow easy dev setup
* database
  * Set up environment databases
  * Backup and restore scripts
  * standard backup filename and location
  * daily snapshots of development database
      * (kept for 1 week)
  * simple drush based db restore
* Developer access
  * ssh to dev server
  * ssh to prod server from (dev server?)
  * access to log files (dev and prod)
* Email prevention
  * MTA should route all email to a single email address
      * Much safer at a system level than a drupal level
* documentation
--
===============================================================
Code Positive Ltd.                              Drupal +
http://codepositive.com
 Skills.Networks.Process.Development
Office: 0207 987 3928
Mobile: 07971 478 482
Skype: the-greenman
twitter: @greenman

hours, totalhours changed

chris — Mon, 29 Jun 2015 09:15:34 GMT

hours changed from 0 to 0.25
totalhours changed from 0.35 to 0.6

Thanks for the write up above, I'm not totally clear regarding exactly what things I'll be needed for, I presume you would like me to do the following?

Create SSH accounts
Set up Nginx
Set up MySQL databases
Set up DNS
Set up MTA
Set up Munin
Document the setup

I assume that Peter will do all the deployment scripts and Solr setup? I would expect that above 7 tasks shouldn't take more that 7 hours.

Note that backups are being sorted out on ticket:11 and there is also a possible issues with Drush, ticket:18.

I'm not sure that 1GB of RAM would be essential for the dev server, it'll need to have at least 1 Nginx, php5-fpm and Java process running, looking at the Crin2 Munin stats, this is the amount of memory 1 process takes (dividing the memory used by the number of processes, of course this isn't totally accurate due to shared memory between processes):

php5-fpm - 75M
nginx - 10M
java - 140M

I suspect that a 512M RAM dev server would be adequate, note that upgrading a virtual server from 512M to 1G is possible whereas downgrading from 1G to 512M isn't.

peter — Mon, 29 Jun 2015 14:50:20 GMT

Ok. Chris, lets try the 512M server config. I would also need a little help from from you on the webserver umask and ownership permissions.

peter — Mon, 29 Jun 2015 15:35:05 GMT

Hi Chris
What is your cost estimate for a server with this spec?
On 29 June 2015 at 15:50, CRIN Trac <trac@trac.crin.org> wrote:
> #23: Development and staging environment
> ------------------------------------+-----------------------------------
>                  Reporter:  chris   |                Owner:  chris
>                      Type:  defect  |               Status:  new
>                  Priority:  major   |            Milestone:  Maintenance
>                 Component:  drupal  |              Version:
>                Resolution:          |             Keywords:
> Estimated Number of Hours:  0       |  Add Hours to Ticket:  0
>                 Billable?:  1       |          Total Hours:  0.6
> ------------------------------------+-----------------------------------
>
> Comment (by peter):
>
>  Ok. Chris, lets try the 512M server config.
>  I would also need a little help from from you on the webserver umask and
>  ownership permissions.
>
> --
> Ticket URL: <https://trac.crin.org/trac/ticket/23#comment:6>
> CRIN Trac <https://trac.crin.org/trac>
> Trac project for CRIN website and servers.
>
--
===============================================================
Code Positive Ltd.                              Drupal +
http://codepositive.com
Skills.Networks.Process.Development
Office: 0207 987 3928
Mobile: 07971 478 482
Skype: the-greenman
twitter: @greenman

chris — Mon, 29 Jun 2015 20:59:37 GMT

Replying to peter:

I would also need a little help from from you on the webserver umask and ownership permissions.

No problem.

chris — Mon, 29 Jun 2015 21:00:52 GMT

Replying to peter:

What is your cost estimate for a server with this spec?

The https://1984.is/ Virtual Server prices are here:

https://1984.is/product/vps/

hours, totalhours changed

chris — Tue, 07 Jul 2015 08:39:40 GMT

hours changed from 0 to 0.08
totalhours changed from 0.6 to 0.68

Jenny / Gillian -- would you like me to start setting up a development server for Code Positive as discussed on this ticket above?

If you would, could you (or perhaps ask Andrew to?) order it using the 1984.is management interface:

RAM: 512M
OS: 64 bit Debian 7 (Jessie)
Server Name: CRIN4

And then let me have the login details once it has been setup. Thanks.

hours, totalhours changed

chris — Tue, 07 Jul 2015 10:14:34 GMT

hours changed from 0 to 0.03
totalhours changed from 0.68 to 0.71

Jenny asked by email:

Please could you confirm a quote before we go ahead with this

I guessed that it might take up to 7 hours, details on this comment:

https://trac.crin.org/trac/ticket/23#comment:5

But please note that this is not a quote, it is an estimate of the amount of time it might take.

hours, totalhours changed

chris — Tue, 07 Jul 2015 11:04:29 GMT

hours changed from 0 to 0.1
totalhours changed from 0.71 to 0.81

Following a chat with Andrew I am revising the guess for how long this might take from 1 hour per task listed below to 30 mins per task, so 3.5 hours in total, but please note that there is no slack in this at all so there is no guarantee that this won't prove to be another underestimate of the time needed (I was hoping to avoid underestimating the time needed by estimating 7 hours).

Please also note that no time has been allowed for helping Code Positive with setting up any scripts for deployment and updating of sites and databases or for doing backups.

Replying to chris:

I presume you would like me to do the following?

Create SSH accounts
Set up Nginx
Set up MySQL databases
Set up DNS
Set up MTA
Set up Munin
Document the setup
I would expect that above 7 tasks shouldn't take more that 7 hours.

jenny — Tue, 07 Jul 2015 13:10:05 GMT

Thanks Chris. This looks ok, but we are concerned about the possibility of going over 7 hours as we simply can't afford it. Can we agree this as the maximum?
Thanks,
Jenny
Sent from my iPhone
> On 7 Jul 2015, at 12:04, CRIN Trac <trac@trac.crin.org> wrote:
>
> #23: Development and staging environment
> ------------------------------------+-----------------------------------
>                 Reporter:  chris   |                Owner:  chris
>                     Type:  defect  |               Status:  new
>                 Priority:  major   |            Milestone:  Maintenance
>                Component:  drupal  |              Version:
>               Resolution:          |             Keywords:
> Estimated Number of Hours:  0       |  Add Hours to Ticket:  0.1
>                Billable?:  1       |          Total Hours:  0.71
> ------------------------------------+-----------------------------------
> Changes (by chris):
>
> * hours:  0 => 0.1
> * totalhours:  0.71 => 0.81
>
>
> Comment:
>
> Following a chat with Andrew I am revising the guess for how long this
> might take from 1 hour per task listed below to 30 mins per task, so 3.5
> hours in total, but please note that there is no slack in this at all so
> there is no guarantee that this won't prove to be another underestimate of
> the time needed (I was hoping to avoid underestimating the time needed by
> estimating 7 hours).
>
> Please also note that no time has been allowed for helping Code Positive
> with setting up any scripts for deployment and updating of sites and
> databases or for doing backups.
>
> Replying to [comment:5 chris]:
>> I presume you would like me to do the following?
>>
>> 1. Create SSH accounts
>> 2. Set up Nginx
>> 3. Set up MySQL databases
>> 4. Set up DNS
>> 5. Set up MTA
>> 6. Set up Munin
>> 7. Document the setup
>>
>> I would expect that above 7 tasks shouldn't take more that 7 hours.
>
> --
> Ticket URL: <https://trac.crin.org/trac/ticket/23#comment:12>
> CRIN Trac <https://trac.crin.org/trac>
> Trac project for CRIN website and servers.

hours, totalhours changed

chris — Tue, 07 Jul 2015 13:25:24 GMT

hours changed from 0 to 0.17
totalhours changed from 0.81 to 0.98

Replying to jenny:

Thanks Chris. This looks ok, but we are concerned about the possibility of going over 7 hours as we simply can't afford it. Can we agree this as the maximum?

I wouldn't spend more than 2 or 3 hours on this a day and would document everything I'm doing via Trac so you would be aware of how much time was being spent and how many tasks were being done. The only item in this list that I don't immeditialy know how to resolve is this one (all the others are quite straightforward):

Set up MTA (mail transport agent)

Code Positive have asked for:

Replying to peter:

Email prevention
MTA should route all email to a single email address
Much safer at a system level than a drupal level

If this item were omitted then I would be quite confident that all 6 other tasks could be done in less than 3 hours -- this item is the only, somewhat, unknown task.

Also note that this (rerouting all email to developers rather than clients) can be done at a Drupal level, using, for example, this module:

https://www.drupal.org/project/reroute_email

However not doing this at a MTA level would increase the risk that the development server might send emails to clients, so I wouldn't advise only using the Drupal modules (doing both would probably be a good idea).

jenny — Tue, 07 Jul 2015 13:35:09 GMT

Ok, can we go ahead with this then, please.
Thanks,
Jenny
Sent from my iPhone
> On 7 Jul 2015, at 14:25, CRIN Trac <trac@trac.crin.org> wrote:
>
> #23: Development and staging environment
> ------------------------------------+-----------------------------------
>                 Reporter:  chris   |                Owner:  chris
>                     Type:  defect  |               Status:  new
>                 Priority:  major   |            Milestone:  Maintenance
>                Component:  drupal  |              Version:
>               Resolution:          |             Keywords:
> Estimated Number of Hours:  0       |  Add Hours to Ticket:  0.17
>                Billable?:  1       |          Total Hours:  0.81
> ------------------------------------+-----------------------------------
> Changes (by chris):
>
> * hours:  0 => 0.17
> * totalhours:  0.81 => 0.98
>
>
> Comment:
>
> Replying to [comment:13 jenny]:
>>
>> Thanks Chris. This looks ok, but we are concerned about the possibility
> of going over 7 hours as we simply can't afford it. Can we agree this as
> the maximum?
>
> I wouldn't spend more than 2 or 3 hours on this a day and would document
> everything I'm doing via Trac so you would be aware of how much time was
> being spent and how many tasks were being done. The only item in this list
> that I don't immeditialy know how to resolve is this one (all the others
> are quite straightforward):
>
> * Set up MTA (mail transport agent)
>
> Code Positive have asked for:
>
> Replying to [comment:4 peter]:
>>
>> * Email prevention
>>  * MTA should route all email to a single email address
>>      * Much safer at a system level than a drupal level
>
> If this item were omitted then I would be quite confident that all 6 other
> tasks could be done in less than 3 hours -- this item is the only,
> somewhat, unknown task.
>
> Also note that this (rerouting all email to developers rather than
> clients) can be done at a Drupal level, using, for example, this module:
>
> * https://www.drupal.org/project/reroute_email
>
> However not doing this at a MTA level would increase the risk that the
> development server might send emails to clients, so I wouldn't advise only
> using the Drupal modules (doing both would probably be a good idea).
>
> --
> Ticket URL: <https://trac.crin.org/trac/ticket/23#comment:14>
> CRIN Trac <https://trac.crin.org/trac>
> Trac project for CRIN website and servers.

hours, totalhours changed

chris — Tue, 07 Jul 2015 13:40:00 GMT

hours changed from 0 to 0.02
totalhours changed from 0.98 to 1.0

Replying to jenny:

Ok, can we go ahead with this then, please.

Would you like me to include setting up the MTA or would you be happy for that to be omitted?

Are you happy to order the virtual server and let me have the details, as per:

Replying to chris:

could you (or perhaps ask Andrew to?) order it using the 1984.is management interface:

RAM: 512M
OS: 64 bit Debian 7 (Jessie)
Server Name: CRIN4
And then let me have the login details once it has been setup.

chris — Tue, 07 Jul 2015 13:41:26 GMT

Sorry, I got the version number wrong, Jessie is Debian 8 not 7:

Replying to chris:

OS: 64 bit Debian 7 (Jessie)

peter — Tue, 07 Jul 2015 13:50:04 GMT

Hi Chris
Are we going to be using exim? If so I can take on the mail routing.
I have a router for exim that would work to reroute all email.
Peter
On Tue, 7 Jul 2015 14:41 CRIN Trac <trac@trac.crin.org> wrote:
> #23: Development and staging environment
> ------------------------------------+-----------------------------------
>                  Reporter:  chris   |                Owner:  chris
>                      Type:  defect  |               Status:  new
>                  Priority:  major   |            Milestone:  Maintenance
>                 Component:  drupal  |              Version:
>                Resolution:          |             Keywords:
> Estimated Number of Hours:  0       |  Add Hours to Ticket:  0
>                 Billable?:  1       |          Total Hours:  1.0
> ------------------------------------+-----------------------------------
>
> Comment (by chris):
>
>  Sorry, I got the version number wrong, Jessie is Debian 8 not 7:
>
>  Replying to [comment:16 chris]:
>  > >
>  > > * OS:  64 bit Debian 7 (Jessie)
>
> --
> Ticket URL: <https://trac.crin.org/trac/ticket/23#comment:17>
> CRIN Trac <https://trac.crin.org/trac>
> Trac project for CRIN website and servers.
>

hours, totalhours changed

chris — Tue, 07 Jul 2015 13:53:26 GMT

hours changed from 0 to 0.02
totalhours changed from 1.0 to 1.02

Replying to peter:

Are we going to be using exim? If so I can take on the mail routing.

I have a router for exim that would work to reroute all email.

Excellent, yes, I was just going to use the Debian default, Exim, I'm happy to leave the configuration of that to you. Note that we should ensure that Exim is configured before we install copies of the site.

hours, totalhours changed

chris — Mon, 13 Jul 2015 08:38:31 GMT

hours changed from 0 to 0.02
totalhours changed from 1.02 to 1.04

Is the dev server going to be ordered this week? I'm asking since I'm going to have reduced availability from the end of this week for 6 weeks.

cc, status changed

chris — Mon, 13 Jul 2015 13:16:19 GMT

cc mori added
status changed from new to accepted

Mori added as a Cc for this ticket.

hours, totalhours changed

chris — Mon, 13 Jul 2015 17:56:01 GMT

hours changed from 0 to 0.25
totalhours changed from 1.04 to 1.29

SUMMARY: The following issue has been resolved and Jenny and Gillian don't need to take any action on it.

Request from Mori:

In order to automatically change the site's configuration per environment, we need nginx to have environment variables available for PHP to know which environment the site is running on.

It will be good if we can refer to the variable like below:
$_SERVER['SITE_ENV']
If possible, please make the return values as below:

Prod: 'crin_prod'
Stage: 'crin_stage'
Dev: 'crin_dev'

I have added this to /etc/nginx/sites-available/crin.org on Crin2:

  fastcgi_param SITE_ENV crin_prod;

Restarted nginx and php5-fpm and created a test .php file containing:

<?php
phpinfo();
?>

And _SERVER["SITE_ENV"] was set to crin_prod, so this will be fine on the dev server, it'll just be a matter of coding the other values into the nginx config files.

hours, totalhours changed

chris — Tue, 14 Jul 2015 13:15:09 GMT

hours changed from 0 to 0.79
totalhours changed from 1.29 to 2.08

The server has been created, and I have completed one of the tasks, setting up ssh, details follow.

I'm sorry that this has taken longer than the estimate of 30 mins.

SSH config

Adding user accounts for chris, peter and mori:

export NEWUSER="chris"
adduser --disabled-password $NEWUSER
adduser $NEWUSER sudo
mkdir /home/$NEWUSER/.ssh
chmod 700 /home/$NEWUSER/.ssh
chown -R $NEWUSER:$NEWUSER  /home/$NEWUSER/.ssh

Install vim and set it to be the default editor:

aptitude install vim
echo "export EDITOR='vim'" >> /root/.bashrc
source /root/.bashrc
echo "syntax on" >> /root/.vimrc

Edit /etc/sudoers to enable password less sudo:

#%sudo  ALL=(ALL:ALL) ALL
%sudo  ALL=(ALL)       NOPASSWD: ALL

Add the root ssh public keys to the server from Crin1 and Crin2 and lock them down to the specific IP addresses:

mkdir ~/.ssh
vi ~/.ssh/authorized_keys

Eg, /root/.ssh/authorized_keys:

from="93.95.228.179" ssh-rsa AAA... root@CRIN1
from="93.95.228.180" ssh-rsa AAA... root@CRIN2

Edit /root/ssh/config on Crin1 and Crin2 to add:

Host crin4
  User root
  Hostname crin4.crin.org

Test the ssh connections from Crin1 and Crin2:

ssh crin4

Copy public keys from Crin1 to Crin4:

scp /home/chris/.ssh/authorized_keys crin4:/home/chris/.ssh/
scp /home/peter/.ssh/authorized_keys crin4:/home/peter/.ssh/
scp /home/mori/.ssh/authorized_keys crin4:/home/mori/.ssh/

Chown them on Crin4:

chown chris:chris /home/chris/.ssh/authorized_keys
chown peter:peter /home/peter/.ssh/authorized_keys
chown mori:mori /home/mori/.ssh/authorized_keys

Add DNS entries:

crin4 900 IN A 93.95.228.222
*.crin4 900 IN A 93.95.228.222
dev 900 IN A 93.95.228.222
stage 900 IN A 93.95.228.222

Edit /etc/hosts on Crin1, Crin2 and Crin4:

93.95.228.222   crin4 crin4.crin.org dev.crin.org dev stage.crin.org stage
93.95.228.179   crin1 crin1.crin.org phpmyadmin.crin.org crin1.webarch.net cloud.crin.org www.cloud.crin.org stats.crin.org wiki.crin1.crin.org wiki.crin.org
93.95.228.180   crin2 crin2.crin.org crin2.webarch.net

Edit /etc/hostname:

crin4.crin.org

And /etc/mailname:

crin4.crin.org

Disable logins via passwords (console logins will still work), edit /etc/ssh/sshd_config and change:

AllowGroups sudo root
PasswordAuthentication no

Restart ssh and test -- it's working fine.

hours, totalhours changed

chris — Tue, 14 Jul 2015 13:17:48 GMT

hours changed from 0 to 0.03
totalhours changed from 2.08 to 2.11

Actually I haven't gone over time -- in ticket:23#comment:23 I configured ssh and did the DNS setup, so I'm, so far, under time for the first two tasks.

hours, totalhours changed

chris — Tue, 14 Jul 2015 14:00:12 GMT

hours changed from 0 to 0.67
totalhours changed from 2.11 to 2.78

Jenny / Gillian, I'm afraid the original estimate of 1 hour for setting up nginx was more accurate (but also quite a big underestimate) than the revised estimate of 30 mins -- I have spent 40 mins setting up nginx but have not completed the task -- how would you like me to proceed?

Nginx

Install some packages, based on the list of things installed on Crin2 on ticket:6#comment:4 (omitting drush as Crin2 has one from source, see ticket:6#comment:7 and adding rsync).

aptitude install nginx-common nginx-extras php5 php5-fpm php-pear php5-mysql php5-intl php5-imagick php5-memcached memcached rsync

Copy the live config over from Crin2:

rsync -av /etc/nginx/sites-available/ crin4:/etc/nginx/sites-available/

Copy the vim config for syntax highlighting over:

rsync -av /root/.vim/ crin4:/root/.vim/

Delete the config files we don't need, copy and rename the others:

cd /etc/nginx/sites-available
rm crin2.crin.org
rm crin.com
mv enoc.crin.org enoc.crin4.crin.org
mv solr.crin.org solr.crin4.crin.org
cp crin.org dev.crin.org
mv crin.org stage.crin.org

Set up https://cacert.org/ cert following the steps done for Crin1 on ticket:8#comment:2:

mkdir /root/bin
cd /root/bin
wget http://svn.cacert.org/CAcert/Software/CSRGenerator/csr
chmod 700 csr
./csr
Private Key and Certificate Signing Request Generator
This script was designed to suit the request format needed by
the CAcert Certificate Authority. www.CAcert.org
Short Hostname (ie. imap big_srv www2): crin4
FQDN/CommonName (ie. www.example.com) : crin4.crin.org
Type SubjectAltNames for the certificate, one per line. Enter a blank line to finish
SubjectAltName: DNS:crin4.crin.org
SubjectAltName: DNS:*.crin4.crin.org
SubjectAltName: DNS:dev.crin.org
SubjectAltName: DNS:stage.crin.org
SubjectAltName: DNS:crin4.webarch.net
SubjectAltName: DNS:*.crin4.webarch.net
SubjectAltName: DNS:
Running OpenSSL...

Edit the config, dev.crin.org:

server {
        #listen 80;
        listen 80 default_server;
        server_name dev.crin.org;
        root /var/www/dev;
        access_log /var/log/nginx/dev.crin.org.access.log;
        error_log  /var/log/nginx/dev.crin.org.error.log info;
        index index.php;
        # only include for unencrypted site
        include  gzip;
        location = /favicon.ico {
                log_not_found off;
                access_log off;
        }
        location = /robots.txt {
                allow all;
                log_not_found off;
                access_log off;
        }
        # taken from .htaccess file
        location ~* \.(engine|inc|info|install|make|module|profile|test|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)(~|\.sw[op]|\.bak|\.orig|\.save)?$|^(\..*|Entries.*|Repository|Root|Tag|Template)$|^#.*#$|\.php(~|\.sw[op]|\.bak|\.orig\.save)$ {
                #allow 127.0.0.1;
                deny all;
        }
        # Very rarely should these ever be accessed outside of your lan
        location ~* \.(txt|log)$ {
                #allow 127.0.0.1;
                deny all;
        }
        # This matters if you use drush
        location = /backup {
                deny all;
        }
        # hide backup_migrate files
        location ~* ^/files/backup_migrate {
                deny all;
        }
        location ~* ^/sites/default/files/backup {
                deny all;
        }
        location ~ \..*/.*\.php {
                return 403;
        }
        # No no for private
                location ~ ^/sites/.*/private/ {
                return 403;
        }
        # this isn't readable anyway
                location ~ ^/xmlrpc\.php$ {
                return 403;
        }
        # https://groups.drupal.org/node/238983
        location ~* /sites/default/files/.*\.php$ {
                return 444;
        }
        # symlink to sites/default/files/images/i
        location ~* /i/.*\.php$ {
                return 444;
        }
        # symlink to sites/default/files/images/docs
        location ~* /docs/.*\.php$ {
                return 444;
        }
        # Block access to "hidden" files and directories whose names begin with a
        # period. This includes directories used by version control systems such
        # as Subversion or Git to store control files.
        location ~ (^|/)\. {
                return 403;
        }
#added
        # login redirect
        location ~ /user {
                rewrite ^/(.*)$ https://dev.crin.org/$1? permanent;
                #rewrite ^/(.*)$ https://$server_name/$1? permanent;
                #rewrite ^/(.*)$ https://crin.web1.crin.webarch.net/$1? permanent;
        }
        ### Upload progress support.
        ### http://drupal.org/project/filefield_nginx_progress
        ### http://github.com/masterzen/nginx-upload-progress-module
        location ~ (?<upload_form_uri>.*)/x-progress-id:(?<upload_id>\d*) {
                rewrite ^ $upload_form_uri?X-Progress-ID=$upload_id;
        }
        location ^~ /progress {
                upload_progress_json_output;
                report_uploads uploads;
        }
        # Restrict cron access
        # http://mailman.nginx.org/pipermail/nginx/2010-August/022009.html
        location /cron.php {
                allow 127.0.0.1;
                allow 93.95.228.222;
                error_page 403 =404;
                fastcgi_pass unix:/var/run/php5-fpm.sock;
                deny all;
        }
        location / {
                # This is cool because no php is touched for static content
                try_files $uri $uri/ @rewrite;
                error_page 404 = @rewrite;
                expires max;
        }
        location @rewrite {
                # Some modules enforce no slash (/) at the end of the URL
                # Else this rewrite block wouldn&amp;#39;t be needed (GlobalRedirect)
                rewrite ^/(.*)$ /index.php?q=$1;
        }
        location ~ \.php$ {
                include fastcgi_params;
                fastcgi_param SITE_ENV crin_dev;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_intercept_errors on;
                fastcgi_send_timeout 1200s;
                fastcgi_read_timeout 1200s;
                fastcgi_buffers 256 16k;
                fastcgi_busy_buffers_size 32k;
                fastcgi_pass unix:/var/run/php5-fpm.sock;
                # track uploads in the 'uploads' zone
                # remember connections for 30s after they finished
                track_uploads uploads 60s;
        }
}
server {
        listen 80;
        server_name dev.crin.org;
        access_log /var/log/nginx/dev.crin.org.access.log;
        error_log  /var/log/nginx/dev.crin.org.error.log info;
        location / {
                return      301 http://dev.crin.org$request_uri;
        }
}
server {
        listen 443 ssl spdy default_server;
        #server_name crin.web1.crin.webarch.net crin.web1;
        server_name dev.crin.org;
        root /var/www/drupal;
        ssl  on;
        #ssl_certificate     /etc/ssl/gandi/crin.org.chained.pem;
        #ssl_certificate_key /etc/ssl/gandi/crin.org.key.pem;
        ssl_certificate  /etc/ssl/cacert/crin4.crin.org.chained.pem;
        ssl_certificate_key  /etc/ssl/cacert/crin4.crin.org.key.pem;
        ssl_dhparam /etc/ssl/gandi/dhparam.pem;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA;
        ssl_prefer_server_ciphers on;
        add_header Strict-Transport-Security max-age=15768000;
        # 24 hours
        #add_header Strict-Transport-Security max-age=86400;
        ## Use a SSL/TLS cache for SSL session resume.
        ssl_session_cache shared:SSL:60m;
        ssl_session_timeout 30m;
        # see https://wiki.mozilla.org/Security/Server_Side_TLS#Nginx
        # https://developer.mozilla.org/en-US/docs/HTTP/X-Frame-Options
        add_header X-Frame-Options SAMEORIGIN;
        # OCSP Stapling -- this needs a newer version of Nginx
        # http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling
        # https://packages.debian.org/wheezy-backports/nginx-extras
        # fetch OCSP records from URL in ssl_certificate and cache them
        #ssl_stapling on;
        #ssl_stapling_verify on;
        ## verify chain of trust of OCSP response using Root CA and Intermediate certs
        #ssl_trusted_certificate /etc/ssl/gandi/gandi.pem;
        access_log /var/log/nginx/dev.crin.org.ssl_access.log;
        error_log  /var/log/nginx/dev.org.ssl_error.log info;
        index index.php;
#location = /owncloud {
#
#allow all;
#}
        location = /favicon.ico {
                log_not_found off;
                access_log off;
        }
        location = /robots.txt {
                allow all;
                log_not_found off;
                access_log off;
        }
        location = /852C9C97D0C38497FD3777A798844720.txt {
                allow all;
        }
        # taken from .htaccess file
        location ~* \.(engine|inc|info|install|make|module|profile|test|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)(~|\.sw[op]|\.bak|\.orig|\.save)?$|^(\..*|Entries.*|Repository|Root|Tag|Template)$|^#.*#$|\.php(~|\.sw[op]|\.bak|\.orig\.save)$ {
                #allow 127.0.0.1;
                deny all;
        }
        # Very rarely should these ever be accessed outside of your lan
        location ~* \.(txt|log)$ {
                #allow 127.0.0.1;
                deny all;
        }
        # This matters if you use drush
        location = /backup {
                deny all;
        }
        # hide backup_migrate files
                location ~* ^/files/backup_migrate {
                deny all;
        }
        location ~* ^/sites/default/files/backup {
                deny all;
        }
        # not
        location ~ \..*/.*\.php {
                return 403;
        }
        # No no for private
                location ~ ^/sites/.*/private/ {
                return 403;
        }
        # this isn't readable anyway
                location ~ ^/xmlrpc\.php$ {
                return 403;
        }
        # https://groups.drupal.org/node/238983
        location ~* /sites/default/files/.*\.php$ {
                return 444;
        }
        # symlink to sites/default/files/images/i
        location ~* /i/.*\.php$ {
                return 444;
        }
        # symlink to sites/default/files/images/docs
        location ~* /docs/.*\.php$ {
                return 444;
        }
        # Block access to "hidden" files and directories whose names begin with a
        # period. This includes directories used by version control systems such
        # as Subversion or Git to store control files.
        location ~ (^|/)\. {
                return 403;
        }
        ### Upload progress support.
        ### http://drupal.org/project/filefield_nginx_progress
        ### http://github.com/masterzen/nginx-upload-progress-module
        location ~ (?<upload_form_uri>.*)/x-progress-id:(?<upload_id>\d*) {
                rewrite ^ $upload_form_uri?X-Progress-ID=$upload_id;
        }
        location ^~ /progress {
                upload_progress_json_output;
                report_uploads uploads;
        }
        # Restrict cron access
        # http://mailman.nginx.org/pipermail/nginx/2010-August/022009.html
        location /cron.php {
                allow 127.0.0.1;
                allow 93.95.228.222;
                error_page 403 =404;
                fastcgi_pass unix:/var/run/php5-fpm.sock;
                deny all;
        }
        location / {
                # This is cool because no php is touched for static content
                try_files $uri $uri/ @rewrite;
                error_page 404 = @rewrite;
                expires max;
        }
        location @rewrite {
                # Some modules enforce no slash (/) at the end of the URL
                # Else this rewrite block wouldn&amp;#39;t be needed (GlobalRedirect)
                rewrite ^/(.*)$ /index.php?q=$1;
        }
        location ~ \.php$ {
                include fastcgi_params;
                fastcgi_param SITE_ENV crin_dev;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_intercept_errors on;
                fastcgi_send_timeout 1200s;
                fastcgi_read_timeout 1200s;
                fastcgi_buffers 256 16k;
                fastcgi_busy_buffers_size 32k;
                fastcgi_pass unix:/var/run/php5-fpm.sock;
                # track uploads in the 'uploads' zone
                # remember connections for 30s after they finished
                track_uploads uploads 60s;
        }
}

hours, totalhours changed

chris — Tue, 14 Jul 2015 18:05:18 GMT

hours changed from 0 to 0.08
totalhours changed from 2.78 to 2.86

Jenny / Gillian -- I have been thinking about the following, the estimate of 3 hours was based on these 6. tasks (see ticket:23#comment:5 and ticket:23#comment:14):

Create SSH accounts
Set up Nginx
Set up MySQL databases
Set up DNS
Set up Munin
Document the setup

It is possible that setting up Munin and MySQL will take less than an hour, so I'll do them next and see where were are up to. We could also omit documenting the server, although this might cause things to take longer in the long run (more time will be spent trying to work out how things are setup) it would save some time in the short term and the live servers were not properly documented to save time (see Crin1 and Crin2) and all the answers can be found in the tickets comments with enough searching.

How does that sound as a way forward?

Replying to chris:

Jenny / Gillian, I'm afraid the original estimate of 1 hour for setting up nginx was more accurate (but also quite a big underestimate) than the revised estimate of 30 mins -- I have spent 40 mins setting up nginx but have not completed the task -- how would you like me to proceed?

gillian — Wed, 15 Jul 2015 09:40:04 GMT

Hi Chris,
We should put documentation in place. Please check in once you have reached
3 hours.
Best,
Gillian
On 14 July 2015 at 19:05, CRIN Trac <trac@trac.crin.org> wrote:
> #23: Development and staging environment
> ------------------------------------+-----------------------------------
>                  Reporter:  chris   |                Owner:  chris
>                      Type:  defect  |               Status:  accepted
>                  Priority:  major   |            Milestone:  Maintenance
>                 Component:  drupal  |              Version:
>                Resolution:          |             Keywords:
> Estimated Number of Hours:  0       |  Add Hours to Ticket:  0.08
>                 Billable?:  1       |          Total Hours:  2.78
> ------------------------------------+-----------------------------------
> Changes (by chris):
>
>  * hours:  0 => 0.08
>  * totalhours:  2.78 => 2.86
>
>
> Comment:
>
>  Jenny / Gillian -- I have been thinking about the following, the estimate
>  of 3 hours was based on these 6. tasks (see ticket:23#comment:5 and
>  ticket:23#comment:14):
>
>  1. Create SSH accounts
>  2. Set up Nginx
>  3. Set up MySQL databases
>  4. Set up DNS
>  5. Set up Munin
>  6. Document the setup
>
>  It is possible that setting up Munin and MySQL will take less than an
>  hour, so I'll do them next and see where were are up to. We could also
>  omit documenting the server, although this might cause things to take
>  longer in the long run (more time will be spent trying to work out how
>  things are setup) it would save some time in the short term and the live
>  servers were not properly documented to save time (see [[Crin1]] and
>  [[Crin2]]) and all the answers can be found in the tickets comments with
>  enough searching.
>
>  How does that sound as away forward?
>
>  Replying to [comment:25 chris]:
>  > Jenny / Gillian, I'm afraid the original estimate of 1 hour for setting
>  up `nginx` was more accurate (but also quite a big underestimate) than the
>  revised estimate of 30 mins -- I have spent 40 mins setting up `nginx` but
>  have not completed the task -- how would you like me to proceed?
>
> --
> Ticket URL: <https://trac.crin.org/trac/ticket/23#comment:26>
> CRIN Trac <https://trac.crin.org/trac>
> Trac project for CRIN website and servers.
>
--
Gillian Harrow
Organisational Development Manager
*Child Rights International Network - CRIN*
Unit W125-127, Westminster Business Square
1-45 Durham Street
London SE11 5JH
United Kingdom
E: gillian@crin.org
T: +44 (0)20 7401 2257
Website: www.crin.org
Twitter: @CRINwire

hours, totalhours changed

chris — Wed, 15 Jul 2015 10:11:23 GMT

hours changed from 0 to 0.07
totalhours changed from 2.86 to 2.93

Replying to gillian:

We should put documentation in place.

OK, great, apart from documenting the dev server work is also needed to document the live servers and also I think we could do with documentation about how best to use Trac.

Please check in once you have reached 3 hours.

The total for this ticket was, before I wrote this comment, 2h 51m, see: ticket:23#comment:26, before I started work on the dev server yesterday it stood at 1h 17m, see ticket:23#comment:22.

hours, totalhours changed

chris — Wed, 15 Jul 2015 10:50:39 GMT

hours changed from 0 to 0.5
totalhours changed from 2.93 to 3.43

Gillian / Jenny: The Munin install has taken 30 mins, however it has yet to be tested and some services won't be producing graphs as they have yet to be installed and configured.

Munin

Setting up Munin, referencing ticket:10, install the client on Crin4:

aptitude install munin-node

Backup the config file on Crin4:

mv /etc/munin/plugin-conf.d/munin-node /etc/munin/plugin-conf.d/munin-node.orig

Copy the config from Crin2:

scp /etc/munin/plugin-conf.d/munin-node crin4:

Copy plugins from Crin2, on Crin4:

mkdir -p /usr/local/share/munin/plugins/php5-fpm-munin-plugins/

On Crin2:

rsync -av /usr/local/share/munin/plugins/ crin4:/usr/local/share/munin/plugins/

Enable the same plugins on Crin4 as on Crin2:

cd /etc/munin/plugins
ln -s /usr/share/munin/plugins/fail2ban
ln -s /usr/share/munin/plugins/fw_conntrack
ln -s /usr/share/munin/plugins/fw_forwarded_local
ln -s /usr/share/munin/plugins/ip_ ip_93.95.228.222
ln -s /usr/share/munin/plugins/memcached_ memcached_bytes
ln -s /usr/share/munin/plugins/memcached_ memcached_counters
ln -s /usr/share/munin/plugins/memcached_ memcached_rates
ln -s /usr/share/munin/plugins/multips
ln -s /usr/share/munin/plugins/multips_memory
ln -s /usr/share/munin/plugins/nginx_request
ln -s /usr/share/munin/plugins/nginx_status
rm nfs4_client nfs_client nfsd nfsd4
aptitude remove nfs-common
ln -s /usr/share/munin/plugins/ntp_kernel_err
ln -s /usr/share/munin/plugins/ntp_kernel_pll_freq
ln -s /usr/share/munin/plugins/ntp_kernel_pll_off
ln -s /usr/share/munin/plugins/ntp_offset
aptitude install npt
ln -s /usr/local/share/munin/plugins/php5-fpm-munin-plugins/phpfpm_average
ln -s /usr/local/share/munin/plugins/php5-fpm-munin-plugins/phpfpm_connections
ln -s /usr/local/share/munin/plugins/php5-fpm-munin-plugins/phpfpm_memory
ln -s /usr/local/share/munin/plugins/php5-fpm-munin-plugins/phpfpm_processes
ln -s /usr/local/share/munin/plugins/php5-fpm-munin-plugins/phpfpm_status
ln -s /usr/local/share/munin/plugins/php_opcache_memoryusage
ln -s /usr/local/share/munin/plugins/php_opcache_restarts

On Crin1 add the following to /etc/munin/munin.conf:

[crin4.crin.org]
     address 93.95.228.222
     use_node_name yes

On Crin4 edit the following lines in /etc/munin/munin-node.conf:

#allow ^127\.0\.0\.1$
#allow ^::1$
allow ^93\.95\.228\.179$
#host *
host 93.95.228.222

Restart the client on Crin4:

service munin-node restart

We should soon have graphs at this address:

https://munin.crin.org/munin/crin.org/crin4.crin.org/index.html

Note that some things won't yet work as the applications haven't been installed, a couple I can think of are the firewall and memcache.

hours, totalhours changed

chris — Wed, 15 Jul 2015 11:32:56 GMT

hours changed from 0 to 0.54
totalhours changed from 3.43 to 3.97

Gillian / Jenny, MySQL has been set up for the dev server, copies of the live database haven't been copied yet, this task has taken just over 30 min.

MySQL

Copy config files from Crin2 to Crin4:

scp /root/.my.cnf crin4:
scp /home/peter/.my.cnf crin4:/home/peter/
scp /home/mori/.my.cnf crin4:/home/mori/
scp /home/chris/.my.cnf crin4:/home/chris/

On Crin4 chown files copied and create a directory for the https://cacert.org/ keys and certs:

chown peter:peter /home/peter/.my.cnf
chown mori:mori /home/mori/.my.cnf
chown chris:chris /home/chris/.my.cnf
mkdir /etc/ssl/cacert

Copy keys and certs from Crin1:

rsync -av /etc/ssl/cacert/ crin4:/etc/ssl/cacert/

On Crin1 add databases and users for the stage and dev sites, following the steps documented for Crin2, ticket:6#comment:4:

sudo -i
mysql
mysql> CREATE DATABASE stage;
  Query OK, 1 row affected (0.01 sec)
mysql> GRANT ALL ON stage.* to 'stage'@'crin4' identified by 'XXX' ' REQUIRE SSL;
  Query OK, 0 rows affected (0.04 sec)
mysql> CREATE DATABASE dev;
  Query OK, 1 row affected (0.02 sec)
mysql> GRANT ALL ON dev.* to 'dev'@'crin4' identified by 'XXX' REQUIRE SSL;
  Query OK, 0 rows affected (0.00 sec)
mysql> FLUSH PRIVILEGES;
  Query OK, 0 rows affected (0.07 sec)

Edit the firewall on Crin1 to allow MySQL connections from Crin4, add the following to /etc/iptables/rules.v4:

-A INPUT -p tcp -s 93.95.228.222 --dport 3306 -j ACCEPT

Reload the firewall on Crin1 and check:

iptables-restore < /etc/iptables/rules.v4
iptables -L | grep crin4
  ACCEPT     tcp  --  crin4                anywhere             tcp dpt:mysql

Check what MySQL package are installed on Crin2:

aptitude search mysql | grep ^i
i A libdbd-mysql-perl               - Perl5 database interface to the MySQL data
i A libmysqlclient18                - MySQL database client library
i   mysql-client-5.5                - MySQL database client binaries
i A mysql-common                    - MySQL database common files, e.g. /etc/mys
i   php5-mysql                      - MySQL module for php5

Install these MySQL packages on Crin4:

aptitude install libdbd-mysql-perl libmysqlclient18 mysql-client-5.5 mysql-common php5-mysql

Check the MySQL connection on Crin4:

mysql -ustage -p stage
mysql -udev -p dev

That works, the dev and stage passwords, which will be needed for the settings.php files for the sites, have been saved in /root/mysql.txt on Crin4 (they are also, for now, available in /root/.mysql_history on Crin1).

hours, totalhours changed

chris — Wed, 15 Jul 2015 11:44:12 GMT

hours changed from 0 to 0.17
totalhours changed from 3.97 to 4.14

The total time for this ticket, as of the last comment, ticket:23#comment:30, stood at 3h 58m, before I started work on the server yesterday, ticket:23#comment:23, the total time on this ticket stood at 1h 17m, so the total worked since then was 2h 41m, once this ticket is included I have almost reached the underestimate of 3 hours in total.

I think the main tasks remaining are:

Configure nginx (this has been started, see ticket:23#comment:25)
Configure memcache
Configure php5-fpm
Configure the firewall, iptables and fail2ban
Document the setup of the server

How would you like me to proceed?

hours, totalhours changed

chris — Wed, 15 Jul 2015 12:25:46 GMT

hours changed from 0 to 0.23
totalhours changed from 4.14 to 4.37

One possible issue with the Crin4 server is disk space, it doesn't have room for two full copies of the live site, so far we have 13G of free space:

df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/dm-0        15G  1.2G   13G   9% /
udev             10M     0   10M   0% /dev
tmpfs            99M  4.4M   95M   5% /run
tmpfs           248M     0  248M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           248M     0  248M   0% /sys/fs/cgroup
/dev/sda1       236M   33M  191M  15% /boot

The live site is 20G:

cd /var/www/drupal/
912K    ./misc
2.3M    ./includes
108K    ./profiles
84K     ./scripts
452K    ./stats
19G     ./sites
127M    ./owncloud_old
672K    ./themes
11M     ./modules
16K     ./.idea
381M    ./.git
20G     .

19G of the 20G is in sites/default/files, a couple of options:

If it is essential that both the stage and dev sites have a full copy of all the files then an aditional 30G of disk space will be needed, this would require the server to jump from a 512M RAM VPS to a 1536M RAM VPS (which comes with 48G of disk space), this would cost 3 times as much per month, see https://1984hosting.com/product/vps/
Use the live sites/default/files directory for the dev and stage sites, this could be done various ways, Fuse/SFTP mounting the live date on the dev server (I don't like this idea as it would enable the live site files to be deleted from the dev server) or setting up nginx so that requests for anything in sites/default/files on stage or dev is reverse proxied to the live server, this makes more sense as it would provide the functionality without introducing an additional risk of data loss.

The two options above will need to be discussed with Code Positive. I can't think of a good reason not to do something like the second option suggested above, but there might be other options I haven't thought of.

hours, totalhours changed

chris — Wed, 15 Jul 2015 12:32:28 GMT

hours changed from 0 to 0.03
totalhours changed from 4.37 to 4.4

Gillian / Jenny since yesterday, from ticket:23#comment:23 to ticket:23#comment:32, I have worked 3h 5m on this ticket so I'm stopping work on it until you advise me how you would like me to proceed.

hours, totalhours changed

chris — Fri, 17 Jul 2015 11:18:35 GMT

hours changed from 0 to 1.41
totalhours changed from 4.4 to 5.81

Continuing settings things up...

MySQL

Add /var/www/.my.cnf files for www-data user:

[client]
host=crin1
ssl-cipher=DHE-RSA-AES256-SHA
ssl-ca=/etc/ssl/cacert/cacert.pem
ssl-cert=/etc/ssl/cacert/crin1_cert.pem
ssl-key=/etc/ssl/cacert/crin1_yassl_privatekey.pem

So drush can be run as www-data user.

robots.txt

Create a /var/www/html/robots.txt file containing:

User-agent: *
Disallow: /

And add this to all the server blocks in the Nginx config files in /etc/nginx/sites-available:

        # this site isn't be be indexed
        location = /robots.txt {
                alias /var/www/html/;
        }

And comment:

        #location = /robots.txt {
        #        allow all;
        #        log_not_found off;
        #        access_log off;
        #}

This should ensure search engines don't index the dev server.

HTTPS

Generate a chained cert for Nginx as done on ticket:8#comment:7

mv /root/crin4_* /etc/ssl/cacert/
cat crin4_cert.pem > crin4_cert.chained.pem
cat cacert.pem >> crin4_cert.chained.pem
chown root:www-data crin4_cert.chained.pem

Nginx

Disable the default site and enable the dev site:

cd /etc/nginx/sites-enabled/
rm default
ln -s ../sites-available/dev.crin.org 00-dev.crin.org
service nginx configtest
  [FAIL] Testing nginx configuration: failed!

In the /var/log/nginx/error.log we have:

2015/07/17 10:26:46 [emerg] 30660#0: open() "/etc/nginx/gzip" failed (2: No such file or directory) in /etc/nginx/sites-enabled/00-dev.crin.org:13

So copying that files from Crin2:

scp /etc/nginx/gzip crin4:/etc/nginx/

Testing again:

002:system library:fopen:No such file or directory:fopen('/etc/ssl/cacert/crin4.crin.org.chained.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

Fix the paths:

        ssl_certificate  /etc/ssl/cacert/crin4_cert.chained.pem;
        ssl_certificate_key  /etc/ssl/cacert/crin4_privatekey.pem;
        ssl_dhparam /etc/ssl/cacert/dhparam.pem;

Test again:

2015/07/17 10:35:43 [emerg] 31689#0: BIO_new_file("/etc/ssl/cacert/dhparam.pem") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/ssl/cacert/dhparam.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

Generate the file:

openssl dhparam 2058 -out /etc/ssl/cacert/dhparam.pem

Test again:

2015/07/17 10:42:14 [emerg] 32250#0: zero size shared memory zone "uploads"

On Crin2 we have /etc/nginx/conf.d/upload_progress.conf containing:

upload_progress uploads 100m;

So that files was copied over:

scp conf.d/upload_progress.conf crin4:/etc/nginx/conf.d/

And the configtest passes:

service nginx configtest
[ ok ] Testing nginx configuration:.

So, restarting:

service nginx restart

Seem OK but the /robots.txt file is a 404, trying this suggestion:

location = /robots.txt {
    #expires 30d;
    #add_header Cache-Control public;
    try_files /robots.txt @shared;
}
location @shared {
    root /var/www/html;
}

That still doesn't work, in the logs:

==> /var/log/nginx/dev.crin.org.error.log <==
2015/07/17 11:03:15 [error] 1866#0: *3 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 93.95.228.222, server: dev.crin.org, request: "GET /robots.txt/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "dev.crin.org", referrer: "http://dev.crin.org/"
==> /var/log/nginx/dev.crin.org.access.log <==
93.95.228.222 - - [17/Jul/2015:11:03:15 +0000] "GET /robots.txt/ HTTP/1.1" 404 47 "http://dev.crin.org/" "Mozilla/5.0 (X11; Linux x86_64; rv:39.0) Gecko/20100101 Firefox/39.0"

So the request for /robots.txt is being passed to php5-fpm and I'm not sure why...

hours, totalhours changed

chris — Fri, 17 Jul 2015 12:04:40 GMT

hours changed from 0 to 0.1
totalhours changed from 5.81 to 5.91

It appears my browser was caching a redirect from /robots.txt to /robots.txt/ -- the following works when testing in another browser:

        # this site isn't be be indexed
         location = /robots.txt {
                root /var/www/html;
        }

Also busting the cache with a query string works: http://dev.crin.org/robots.txt?123

hours, totalhours changed

chris — Fri, 17 Jul 2015 21:40:30 GMT

hours changed from 0 to 0.62
totalhours changed from 5.91 to 6.53

Drush

Installing the same version as on the live server, see ticket:6#comment:7

sudo -i
cd /usr/local/src
wget https://github.com/drush-ops/drush/archive/master.zip
unzip master.zip
cd /usr/local/bin/
ln -s ../src/drush-master/drush
mkdir /root/.drush
cp /usr/local/src/drush-master/examples/example.drushrc.php /root/.drush/drushrc.php
mkdir /var/www/.drush
cp /usr/local/src/drush-master/examples/example.drushrc.php /var/www/.drush/drushrc.php
chown -R www-data:www-data /var/www/.drush
cd /usr/local/src
aptitude install curl
curl -sS https://getcomposer.org/installer | php
  #!/usr/bin/env php
  All settings correct for using Composer
  Downloading...
  Composer successfully installed to: /usr/local/src/composer.phar
  Use it: php composer.phar
mv composer.phar /usr/local/bin/composer
aptitude install git
cd /usr/local/src/drush-master
composer install
  Loading composer repositories with package information
  Installing dependencies (including require-dev) from lock file
    - Installing d11wtq/boris (v1.0.10)
      Downloading: 100%
    - Installing pear/console_table (1.2.1)
      Downloading: 100%
    - Installing symfony/var-dumper (v2.7.1)
      Downloading: 100%
    - Installing phpdocumentor/reflection-docblock (2.0.4)
      Downloading: 100%
    - Installing phpunit/php-token-stream (1.4.3)
      Downloading: 100%
    - Installing symfony/yaml (v2.7.1)
      Downloading: 100%
    - Installing sebastian/version (1.0.6)
      Downloading: 100%
    - Installing sebastian/global-state (1.0.0)
      Downloading: 100%
    - Installing sebastian/recursion-context (1.0.0)
      Downloading: 100%
    - Installing sebastian/exporter (1.2.0)
      Downloading: 100%
    - Installing sebastian/environment (1.2.2)
      Downloading: 100%
    - Installing sebastian/diff (1.3.0)
      Downloading: 100%
    - Installing sebastian/comparator (1.1.1)
      Downloading: 100%
    - Installing phpunit/php-text-template (1.2.1)
      Downloading: 100%
    - Installing doctrine/instantiator (1.0.5)
      Downloading: 100%
    - Installing phpunit/phpunit-mock-objects (2.3.4)
      Downloading: 100%
    - Installing phpunit/php-timer (1.0.6)
      Downloading: 100%
    - Installing phpunit/php-file-iterator (1.4.0)
      Downloading: 100%
    - Installing phpunit/php-code-coverage (2.1.6)
      Downloading: 100%
    - Installing phpspec/prophecy (v1.4.1)
      Downloading: 100%
    - Installing phpunit/phpunit (4.7.5)
      Downloading: 100%
    - Installing symfony/process (v2.4.5)
      Downloading: 100%
  pear/console_table suggests installing pear/Console_Color2 (>=0.1.2)
  symfony/var-dumper suggests installing ext-symfony_debug ()
  phpdocumentor/reflection-docblock suggests installing dflydev/markdown (~1.0)
  phpdocumentor/reflection-docblock suggests installing erusev/parsedown (~1.0)
  sebastian/global-state suggests installing ext-uopz (*)
  phpunit/php-code-coverage suggests installing ext-xdebug (>=2.2.1)
  phpunit/phpunit suggests installing phpunit/php-invoker (~1.1)
  Generating autoload files

Test Drupal Install

chown www-data:www-data /var/www/dev/
su - www-data -s /bin/bash
cd dev/
drush dl drupal
mv drupal*/* .
mv drupal*/.htaccess .
mv drupal*/.gitignore .
rmdir drupal*
drush site-install --account-name=$USERNAME --account-pass=$PASSWD_DRUPAL --account-mail=$EMAIL --site-name=$SITENAME --db-prefix=d7_ --db-url=mysql://$USERNAME:$PASSWD@localhost/$USERNAME
  You are about to create a /var/www/dev/sites/default/settings.php file and DROP all tables in your 'dev' database. Do you want to continue? (y/n): y
  Starting Drupal installation. This takes a while. Consider using the --notify global option.       [ok]
  exception 'Exception' with message 'PHP extensions: Disabled                                       [error]
  Drupal requires you to enable the PHP extensions in the following list (see the <a
  href="http://drupal.org/requirements">system requirements page</a> for more information):<div
  class="item-list"><ul><li class="first last">gd</li>
  </ul></div>' in /var/www/dev/includes/install.core.inc:773
aptitude install php5-gd

I then completed the install using the web interface, the settings.php file had to be manually edited for the certificate paths etc:

$databases = array(
  'default' =>
  array (
    'default' =>
    array (
      'database' => 'dev',
      'username' => 'dev',
      'password' => 'XXX',
      'host' => 'crin1',
      'port' => '',
      'driver' => 'mysql',
      'prefix' => '',
      'pdo' => array(
           PDO::MYSQL_ATTR_SSL_KEY => '/etc/ssl/cacert/crin1_yassl_privatekey.pem',
           PDO::MYSQL_ATTR_SSL_CERT => '/etc/ssl/cacert/crin1_cert.pem',
           PDO::MYSQL_ATTR_SSL_CA => '/etc/ssl/cacert/cacert.pem',
        ),
    ),
  ),
);

And the site is working: https://dev.crin.org/ so once the Exim configuration is in place a copy of the live site and it's database should be able to be run.

There are still some tasks to be done, the list in ticket:23#comment:31 and some others like install scripts for some of the Munin plugins.

hours, totalhours changed

chris — Fri, 17 Jul 2015 23:45:52 GMT

hours changed from 0 to 1.66
totalhours changed from 6.53 to 8.19

Munin

Copy scripts from Crin1 to a new /var/www/localhost directory:

rsync -av /var/www/localhost/ crin4:/var/www/localhost/

Enable the Nginx config:

/etc/nginx/sites-enabled
ln -s ../sites-available/localhost 30-localhost
service nginx configtest
  [ ok ] Testing nginx configuration:.
service nginx restart

Test the Munin plugins:

cd /etc/munin/plugins/
munin-run phpfpm_average
  php_average.value 40937472
munin-run phpfpm_connections
  accepted.value U
munin-run phpfpm_memory
  ram.value 81874944
munin-run phpfpm_processes
  php_processes.value 2
munin-run phpfpm_status
  idle.value U
  active.value U
  total.value U

Fixing the ones outputting U, edit /etc/php5/fpm/pool.d/www.conf:

;pm.status_path = /status
pm.status_path = /status

Restart php5-fpm and test again:

cd /etc/munin/plugins/
munin-run phpfpm_connections
  accepted.value 1
munin-run phpfpm_status
  idle.value 1
  active.value 1
  total.value 2
php_opcache_memoryusage
  mem_used.value 5468264
  mem_free.value 61640600
  mem_wasted.value 0
  oom_restarts.value 0
  hash_restarts.value 0
  manual_restarts.value 0
munin-run php_opcache_restarts
  mem_used.value 5468264
  mem_free.value 61640600
  mem_wasted.value 0
  oom_restarts.value 0
  hash_restarts.value 0
  manual_restarts.value 0
munin-run memcached_bytes
  no (Cache::Memcached not found)

Install some perl modules, see ticket:10#comment:8

aptitude install libcache-memcached-perl
munin-run memcached_rates
  memcache_cache_hits.value 0
  memcache_cache_misses.value 0
  memcache_cmd_get.value 0
  memcache_cmd_set.value 0
  memcache_total_connections.value 6
  memcache_total_items.value 0
munin-run memcached_bytes
  memcache_bytes_read.value 14
  memcache_bytes_written.value 1098
munin-run memcached_counters
  memcache_bytes_allocated.value 0
  memcache_curr_connections.value 5
  memcache_curr_items.value 0

Solr

Following ticket:6#comment:8

aptitude install libsolr-java solr-tomcat solr-common

Edit solr.crin4.crin.org:

# default virtual server
server {
        # listen for ipv4
        # http://nginx.org/en/docs/http/ngx_http_core_module.html#listen
        listen   80;
        # server name and server aliases
        # http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name
        server_name solr.crin4.crin.org;
        # Prevent access to any files starting with a dot, like .htaccess
        # or text editor temp files
        location ~ /\. {
                access_log off;
                log_not_found off;
                deny all;
        }
        # Prevent access to tmp files created by vim
        location ~ .~$ {
                return 403;
        }
        # this site isn't be be indexed
        location = /robots.txt {
                root /var/www/html;
        }
        location / {
                rewrite ^/(.*)$ https://solr.crin4.crin.org/$1? permanent;
        }
}
# HTTPS server
#
server {
        #listen   4430;
        listen  443 ssl spdy;
        server_name solr.crin4.crin.org;
        access_log  /var/log/nginx/solr.crin.org.ssl_access.log;
        error_log   /var/log/nginx/solr.crin.org.ssl_error.log notice;
        ssl  on;
        ssl_certificate  /etc/ssl/cacert/crin4_cert.chained.pem;
        ssl_certificate_key  /etc/ssl/cacert/crin4_privatekey.pem;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA;
        ssl_prefer_server_ciphers on;
        #add_header Strict-Transport-Security max-age=31536000;
        # https://developer.mozilla.org/en-US/docs/HTTP/X-Frame-Options
        add_header X-Frame-Options SAMEORIGIN;
        # this site isn't be be indexed
        location = /robots.txt {
                root /var/www/html;
        }
        location / {
                proxy_set_header X-Forwarded-Host $host;
                proxy_set_header X-Forwarded-Server $host;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_pass http://127.0.0.1:8080/;
                satisfy any;
                deny all;
                auth_basic "Solr Admin";
                auth_basic_user_file /var/www/.htpasswd;
        }
}

Enable and test:

cd /etc/nginx/sites-enabled
ln -s ../sites-available/solr.crin4.crin.org 20-solr.crin4.crin.org
service nginx configtest
  [ ok ] Testing nginx configuration:.
service nginx restart

Create a username / passwd for the Solr admin interface:

cd /var/www/
aptitude install apache2-utils
htpasswd -c .htpasswd dev

iptables and fail2ban

Following ticket:2

aptitude install iptables-persistent fail2ban

Copy config files from Crin2:

scp /etc/iptables/rules.v4 crin4:/etc/iptables/
scp /etc/fail2ban/jail.local crin4:/etc/fail2ban/
iptables-restore < /etc/iptables/rules.v4
service fail2ban restart
iptables -L
  Chain INPUT (policy ACCEPT)
  target     prot opt source               destination
  fail2ban-ssh  tcp  --  anywhere             anywhere             multiport dports ssh
             all  --  anywhere             crin2
  ACCEPT     all  --  anywhere             anywhere
  REJECT     all  --  anywhere             loopback/8           reject-with icmp-port-unreachable
  ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
  ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:http
  ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:https
  ACCEPT     tcp  --  crin1                anywhere             tcp dpt:munin
  DROP       tcp  --  anywhere             anywhere             tcp dpt:munin
  ACCEPT     tcp  --  anywhere             anywhere             state NEW tcp dpt:ssh
  ACCEPT     icmp --  anywhere             anywhere             icmp echo-request
  LOG        all  --  anywhere             anywhere             limit: avg 5/min burst 5 LOG level debug prefix "iptables denied: "
  REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
  Chain FORWARD (policy ACCEPT)
  target     prot opt source               destination
  REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
  Chain OUTPUT (policy ACCEPT)
  target     prot opt source               destination
             all  --  crin2                anywhere
  ACCEPT     all  --  anywhere             anywhere
  Chain fail2ban-ssh (1 references)
  target     prot opt source               destination
  RETURN     all  --  anywhere             anywhere

Munin

Checked all the graphs and everything is working: https://munin.crin.org/munin/crin.org/crin4.crin.org/index.html

Nginx

Configure Enoc, /etc/nginx/sites-available/enoc.dev.crin.org and /etc/nginx/sites-available/enoc.stage.crin.org, the DNS will need setting up for these.

Setting up a reverse proxy so files don't have to be copied across, this should still work with locally uploaded files as it first tries for a local files and then does a reverse proxy if it is not found:

        # reverse proxy for files
        location /sites/default/files {
                try_files   $uri @proxy_to_live;
        }
        location @proxy_to_live {
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_redirect off;
                proxy_pass   http://www.crin.org$uri;
        }

This failed with the following in the Nginx log:

2015/07/17 23:24:06 [error] 12969#0: *1 no resolver defined to resolve www.crin.org, client: XX.XX.XX.XX, server: dev.crin.org, request: "GET /sites/default/files/what_we_do.jpg HTTP/1.1", host: "dev.crin.org"

The answer was to add a DNS server to /etc/nginx/nginx.conf:

        resolver 93.95.224.28;

And now we have a file reversed proxied and a local file:

So that means the issue flagged up in ticket:23#comment:32 is resolved -- sites/default/files doesn't need to be copied for each site.

The dev.crin.org was copied to stage.crin.org and edited, s/dev/stage/ and s/ default_server// and enabled:

mkdir /var/www/stage
cd /etc/nginx/sites-enabled
ln -s ../sites-available/stage.crin.org 05-stage.crin.org
service nginx configtest
  [ ok ] Testing nginx configuration:.
service nginx restart

And although there is no test Drupal install the reverse proxy is fine:

https://stage.crin.org/sites/default/files/what_we_do.jpg

That means that the only big task left for me, I think, is documenting how the server is set up at Crin4.

Tasks for Code Positive:

Exim config
Solr, I haven't done anything to the files in /etc/solr/ -- you should probably start by copying the files from Crin2

hours, totalhours changed

chris — Sun, 19 Jul 2015 19:04:33 GMT

hours changed from 0 to 1
totalhours changed from 8.19 to 9.19

Drupal stags.crin.org

Test Drupal install for https://stage.crin.org/

sudo -i
chown www-data:www-data /var/www/stage/ -R
su - www-data -s /bin/bash
cd stage
drush dl drupal
mv drupal*/* .
mv drupal*/.htaccess .
mv drupal*/.gitignore .
rmdir drupal*
drush site-install --account-name=$USERNAME --account-pass=$PASSWD_DRUPAL --account-mail=$EMAIL --site-name=$SITENAME --db-prefix=d7_ --db-url=mysql://$USERNAME:$PASSWD@localhost/$USERNAME

Again the settings.php file needed manually editing as per ticket:23#TestDrupalInstall

Documentation

I have made a start at documenting the server at Crin4 and I think it is read for Code Positive to do some work on it:

There are https://dev.crin.org/ and https://stage.crin.org/ sites set up with a default Drupal 7 install, once Exim has been configured these can be replaced with copies of the site (backup the settings.php file first). If you want to test these sites use drush to generate a login, cd /var/www/dev ; drush uli.
Note that sites/default/files shouldn't be copied onto Crin4 as there isn't disk space for it, Ngnix reverse proxies this path to the live server is a file isn't found locally.
Solr hasn't been configured, but it is installed.
Exim hasn't been configured, but it is installed.
It would be easiest to copy the database on Crin1 as the root user has root access to MySQL, there isn't (intentionally) root MySQL access to the live MySQL server from the dev server.
The live servers, Crin1 and Crin2 have ssh access to Crin4 but not the other way around.

Let me know if anything else needs doing.

component changed

chris — Sun, 19 Jul 2015 19:17:40 GMT

component changed from drupal to crin4

hours, totalhours changed

chris — Mon, 20 Jul 2015 10:25:30 GMT

hours changed from 0 to 0.15
totalhours changed from 9.19 to 9.34

I omitted to chown mori:mori /home/mori/.ssh/authorized_keys so ssh access wasn't working, this has been fixed for mori and peter and I have updated the documentation at wiki:Crin4#sshaccess.

mori — Mon, 20 Jul 2015 11:26:18 GMT

Hi Chris,

We are changing the directory structure of the code to fulfil our requirements around deployment and maintenance. As a result, the nginx config needs to be updated to accommodate this change:

Current structure: Prod: /var/www/drupal/ Dev / Stage: /var/www/[dev|stage]/

New structure Prod: /var/www/drupal/docroot}/ Dev / Stage: /var/www/[dev|stage]/docroot/

Can you please make the above changes?

Also for consistency, can you rename the prod's webroot name from 'drupal' to 'prod'?

hours, totalhours changed

chris — Mon, 20 Jul 2015 11:55:38 GMT

hours changed from 0 to 0.3
totalhours changed from 9.34 to 9.64

Updating the directory structure on Crin2.

We don't need the site to be available at https://crin2.crin.org/ so remove this site:

cd /etc/nginx/sites-enabled
rm 10-crin2.crin.org.conf

Edit /etc/nginx/sites-available/drin.org and /etc/nginx/sites-available/enoc.crin.orgchanging:

1,$s;/drupal;/drupal/docroot;gc

Stop Nginx and move the site:

cd /var/www/
mkdir prod
service nginx stop
mv drupal/ prod/docroot/
service nginx start
service php5-fpm restart
service memcached restart

There is a problem with this, most pages on the live site are now 404's and I don't know why, eg:

https://www.crin.org/en/home/what-we-do

So I think I'll have to undo these changes if I can't find the answer quickly.

chris — Mon, 20 Jul 2015 11:58:12 GMT

Fixed.

jenny — Mon, 20 Jul 2015 12:00:10 GMT

Hi guys,
Our website isn't working. It's coming up with the message file not found.
Is this connected to work you are doing?
Thanks.
Jenny
Sent from my iPhone
> On 20 Jul 2015, at 12:26, CRIN Trac <trac@trac.crin.org> wrote:
>
> #23: Development and staging environment
> ------------------------------------+-----------------------------------
>                 Reporter:  chris   |                Owner:  chris
>                     Type:  defect  |               Status:  accepted
>                 Priority:  major   |            Milestone:  Maintenance
>                Component:  crin4   |              Version:
>               Resolution:          |             Keywords:
> Estimated Number of Hours:  0       |  Add Hours to Ticket:  0
>                Billable?:  1       |          Total Hours:  9.34
> ------------------------------------+-----------------------------------
>
> Comment (by mori):
>
> Hi Chris,
>
> We are changing the directory structure of the code to fulfil our
> requirements around deployment and maintenance. As a result, the nginx
> config needs to be updated to accommodate this change:
>
> '''Current structure:'''
> Prod: {{{/var/www/drupal/}}}
> Dev / Stage: {{{/var/www/[dev|stage]/}}}
>
>
> '''New structure'''
> Prod: {{{/var/www/drupal/docroot}/}}}
> Dev / Stage: {{{/var/www/[dev|stage]/docroot/}}}
>
>
> Can you please make the above changes?
>
> Also for consistency, can you rename the prod's webroot name from 'drupal'
> to 'prod'?
>
> --
> Ticket URL: <https://trac.crin.org/trac/ticket/23#comment:41>
> CRIN Trac <https://trac.crin.org/trac>
> Trac project for CRIN website and servers.

hours, totalhours changed

chris — Mon, 20 Jul 2015 12:09:57 GMT

hours changed from 0 to 0.25
totalhours changed from 9.64 to 9.89

Sorry for the brief downtime, it is now fixed and the live site has been tested.

This regex was wrong:

1,$s;/drupal;/drupal/docroot;gc

It should have been:

1,$s;/drupal;/prod/docroot;gc

On Crin4:

cd /var/www
mv dev docroot
mkdir dev
mv docroot/ dev/
mv stage/ docroot
mkdir stage
mv docroot/ stage/

Edit the Ngnix config and restart services, test the sites:

mori — Mon, 20 Jul 2015 12:18:52 GMT

Hi Chris, please install screen on Crin4. Thanks in advance for your help.

hours, totalhours changed

chris — Mon, 20 Jul 2015 12:29:17 GMT

hours changed from 0 to 0.05
totalhours changed from 9.89 to 9.94

Replying to mori:

please install screen on Crin4.

Done.

mori — Mon, 20 Jul 2015 13:52:21 GMT

Hi Chris, do you have admin access to CRIN's bitbucket account (https://bitbucket.org/crin ) by any chance? If you don't, never mind. If you do, I would like a deployment key to be added to the Bitbucket account. (The origin of the repository cloned on Crin2 is not what we'll be using moving forward.)

Also I've created a user 'bitbucket' on Crin2 and am planning to grant the user read-only access to the repo. If you have alternative suggestions please let me know.

hours, totalhours changed

chris — Mon, 20 Jul 2015 14:33:38 GMT

hours changed from 0 to 0.04
totalhours changed from 9.94 to 9.98

Replying to mori:

do you have admin access to CRIN's bitbucket account (https://bitbucket.org/crin ) by any chance?

I'm afraid not, I didn't even know it existed.

Also I've created a user 'bitbucket' on Crin2 and am planning to grant the user read-only access to the repo. If you have alternative suggestions please let me know.

Sounds fine.

mori — Mon, 20 Jul 2015 21:25:56 GMT

Replying to chris:

I've added 'env.php' to all three environments to check if the env vars are working. Unfortunately it doesn't seem like they are working. Can you please check the setting? Or is there anything wrong with my code?

The file can be found at: /var/www/[environment]/docroot/env.php

hours, totalhours changed

chris — Tue, 21 Jul 2015 10:17:37 GMT

hours changed from 0 to 0.25
totalhours changed from 9.98 to 10.23

Replying to mori:

Or is there anything wrong with my code?

You had ENV_VAR not SITE_ENV, I have edited the dev file:

<?php
if (!empty($_SERVER['ENV_VAR'])) {
  print 'environment: ' . $_SERVER['ENV_VAR'];
}
else {
  print 'not defined';
}
echo ' <p>Server: ' . $_SERVER["SITE_ENV"] . '</p>' ;
//phpinfo();
?>

Results here:

https://dev.crin.org/env.php

I debugged it via phpinfo();.

mori — Tue, 21 Jul 2015 11:22:31 GMT

Replying to chris:

You had ENV_VAR not SITE_ENV

Ah, of course... Thanks Chris.

hours, totalhours changed

chris — Tue, 21 Jul 2015 11:25:47 GMT

hours changed from 0 to 0.03
totalhours changed from 10.23 to 10.26

Replying to mori:

Thanks Chris.

No problem, I have also linked the comments about this from the wiki page: wiki:Crin4#Envvars

status changed; resolution set

chris — Thu, 07 Jan 2016 12:14:21 GMT

status changed from accepted to closed
resolution set to fixed