#6 closed task (fixed)
Migrate Drupal site from GreenQloud
Reported by: | chris | Owned by: | chris |
---|---|---|---|
Priority: | major | Milestone: | Install and configure crin2 |
Component: | drupal | Version: | |
Keywords: | Cc: | jenny, gillian | |
Estimated Number of Hours: | 5 | Add Hours to Ticket: | 0 |
Billable?: | yes | Total Hours: | 7.79 |
Description
Migrate the Drupal site at https://crin.org/ from GreenQloud to crin2.crin.org for Nginx and Solr and Memcache and crin1.crin.org for MySQL.
Attachments (3)
Change History (34)
comment:1 Changed 3 years ago by chris
- Type changed from defect to task
comment:2 Changed 3 years ago by chris
- Milestone changed from Install and configure crin1 to Install and configure crin2
comment:3 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.25
- Total Hours set to 0.25
comment:4 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 1.1
- Total Hours changed from 0.25 to 1.35
PHP packages installed on the current live server:
aptitude search php | grep ^i i A libphp5-embed - HTML-embedded scripting language (Embedded i php-apc - APC (Alternative PHP Cache) module for PHP i A php-console-table - PHP PEAR module to make it easy to build c i php-pear - PEAR - PHP Extension and Application Repos i php-xml-parser - PHP PEAR module for parsing XML i A php5 - server-side, HTML-embedded scripting langu i A php5-cli - command-line interpreter for the php5 scri i A php5-common - Common files for packages built from the p i php5-curl - CURL module for php5 i php5-dev - Files for PHP5 module development i php5-fpm - server-side, HTML-embedded scripting langu i php5-gd - GD module for php5 i A php5-imagick - ImageMagick module for php5 i php5-intl - internationalisation module for php5 i A php5-mcrypt - MCrypt module for php5 i php5-memcached - memcached extension module for PHP5, uses i php5-mysql - MySQL module for php5 i A php5-pgsql - PostgreSQL module for php5 i php5-sqlite - SQLite module for php5 i phpmyadmin - MySQL web administration tool
And nginx:
aptitude search nginx | grep ^i i A nginx-common - small, powerful, scalable web/proxy server i nginx-extras - nginx web/proxy server (extended version)
And memcache:
i A libmemcached10 - C and C++ client library to the memcached i memcached - A high-performance memory object caching s i php5-memcached - memcached extension module for PHP5, uses
Not sure we need the same PHP packages (for example phpmyadmin is running on Crin1 and not needed on Crin2) so installing these packages to start with:
aptitude install nginx-common nginx-extras php5 php5-fpm php-pear php5-mysql php5-intl php5-imagick php5-memcached memcached drush
Copy the key Nginx config files from the live server:
cd /etc/nginx/sites-available scp web1:/etc/nginx/sites-available/crin.com . scp web1:/etc/nginx/sites-available/crin.org . scp web1:/etc/nginx/sites-available/enoc.crin.org .
Copy the SSL/TLS key and cert:
cd /etc/ssl mkdir gandi chmod 700 gandi rsync -av web1:/etc/ssl/gandi/ /etc/ssl/gandi/
Symlink and test Nginx:
cd /etc/nginx/sites-enabled rm default ln -s ../sites-available/crin.org 00-crin.org service nginx configtest [FAIL] Testing nginx configuration: failed!
This is the error in the log:
2015/05/06 13:01:30 [info] 19112#0: Using 32768KiB of shared memory for push module in /etc/nginx/nginx.conf:63 2015/05/06 13:14:22 [emerg] 19856#0: open() "/etc/nginx/gzip" failed (2: No such file or directory) in /etc/nginx/sites-enabled/00-crin.org:13
So:
cd /etc/nginx scp web1:/etc/nginx/gzip . service nginx configtest [FAIL] Testing nginx configuration: failed!
The error this time:
2015/05/06 13:17:10 [emerg] 20015#0: could not build the server_names_hash, you should increase server_names_hash_bucket_size: 32
So these settings were copied into the main /etc/ngin/nginx.conf file from the live server:
server_names_hash_max_size 2048; client_max_body_size 50M;
And we still have :
service nginx configtest [FAIL] Testing nginx configuration: failed!
2015/05/06 13:19:56 [info] 20066#0: Using 32768KiB of shared memory for push module in /etc/nginx/nginx.conf:65 2015/05/06 13:19:56 [emerg] 20066#0: could not build the server_names_hash, you should increase server_names_hash_bucket_size: 32
So this was changed:
server_names_hash_bucket_size 128;
And now the error is:
2015/05/06 13:21:46 [emerg] 20101#0: zero size shared memory zone "uploads"
So the upload_progress.conf file was copied over:
cd /etc/nginx/conf.d scp web1:/etc/nginx/conf.d/upload_progress.conf .
And success:
service nginx configtest [ ok ] Testing nginx configuration:.
Start the services:
service php5-fpm start service nginx start
And as expected we now has a database error as that needs copying and the settings.php will need editing.
On Crin1 create a database and user:
mysql mysql mysql> CREATE DATABASE drupal; mysql> GRANT ALL ON drupal.* to 'drupal'@'crin2' identified by 'XXX' REQUIRE SSL; mysql> FLUSH PRIVILEGES;
On Crin1 create a script to dump the database on db1 and rscnc and import it after enabling ssh access to db1:
#!/bin/bash # sync files rsync -av --exclude "settings.php" web1:/var/www/crin/ /var/www/drupal/ # dump and copy the database ssh db1 "mysqldump -uroot -pXXX crin > /root/crin.sql" scp db1:crin.sql /root/ # import the database cat /root/crin.sql | mysql drupal
The script takes a while to run, it's a 1.4G database file and a 20G site...
I'll continue with this tomorrow.
comment:5 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.05
- Estimated Number of Hours changed from 0 to 5
- Total Hours changed from 1.35 to 1.4
Outstanding things that will need doing on this ticket:
- PHP configuration
- Setting up Memcache
- Setting up Solr
- Testing the copy of the site
- Document the setup at wiki:Drupal
- Final data sync
- Update DNS
So far 1.35 hours have been spent on it, I estimate that it might end up being 6 in total.
After the server goes live there will a need to monitor and adjust server settings, see ticket:9.
comment:6 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.5
- Total Hours changed from 1.4 to 1.9
Note the ~/bin/drupal-db-sync script on Crin1 mentioned in ticket:6#comment:4 has the rsync of the files commented out, this command is in ~/bin/drupal-sync on Crin2:
#!/bin/bash # sync files rsync -av --exclude "settings.php" web1:/var/www/crin/ /var/www/drupal/
The two sync script were run and settings.php was edited on Crin2:
$databases = array ( 'default' => array ( 'default' => array ( 'database' => 'drupal', 'username' => 'drupal', 'password' => 'XXX', 'host' => 'crin1', 'port' => '', 'driver' => 'mysql', 'prefix' => '', 'pdo' => array( PDO::MYSQL_ATTR_SSL_KEY => '/etc/ssl/cacert/crin1_yassl_privatekey.pem', PDO::MYSQL_ATTR_SSL_CERT => '/etc/ssl/cacert/crin1_cert.pem', PDO::MYSQL_ATTR_SSL_CA => '/etc/ssl/cacert/cacert.pem', ), ), ), );
And the site, http://crin2.crin.org/ was checked and we have an error:
PDOException: SQLSTATE[HY000] [2026] SSL connection error: Unable to get certificate in lock_may_be_available() (line 167 of /var/www/drupal/includes/lock.inc).
So, check that we can connect via MySQL on the command line, edit /root/.mysql to:
[client] host=crin1 user = root password = XXX ssl-cipher=DHE-RSA-AES256-SHA ssl-ca=/etc/ssl/cacert/cacert.pem ssl-cert=/etc/ssl/cacert/crin1_cert.pem ssl-key=/etc/ssl/cacert/crin1_yassl_privatekey.pem
On Crin1 allow access from Crin2:
mysql mysql mysql> GRANT ALL ON *.* to 'root'@'crin2' identified by 'XXX' REQUIRE SSL; mysql> FLUSH PRIVILEGES; mysql> exit;
Test on Crin2:
mysql mysql Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 174 Server version: 5.5.43-0+deb8u1 (Debian) Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>
Create a /var/www/drupal/.my.cnf on Crin2:
[client] host=crin1 ssl-cipher=DHE-RSA-AES256-SHA ssl-ca=/etc/ssl/cacert/cacert.pem ssl-cert=/etc/ssl/cacert/crin1_cert.pem ssl-key=/etc/ssl/cacert/crin1_yassl_privatekey.pem
The issue could be that the www-data user doesn't have access to the CAcert cert and key, so on Crin2
sudo -i chown -R root:www-data /etc/ssl/cacert/ chmod 770 /etc/ssl/cacert/ chmod 660 /etc/ssl/cacert/*.pem
And the site is now up and running: http://crin2.crin.org/ -- Jonas can you login and see if things appear to be working OK?
I'll also do some testing.
comment:7 follow-up: ↓ 8 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 1
- Total Hours changed from 1.9 to 2.9
Looked at setting up drush on Crin2 but have hit this issue:
Checking the documentation for the GreenQloud servers and I found that I wrote:
The Debian packaged version of Drush doesn't support MySQL SSL connections so a new version has been installed from github.com in /usr/local/src/drush-6.x and symliked from /usr/local/sbin/
So, download the latest version of Drush on Crin2:
sudo -i cd /usr/local/src https://github.com/drush-ops/drush/archive/master.zip aptitude install unzip unzip master.zip cd /usr/local/bin/ ln -s ../src/drush-master/drush
Set up /root/.drush/drushrc.php and /var/www/drushrc.php:
mkdir /root/.drush cp /usr/local/src/drush-master/examples/example.drushrc.php /root/.drush/drushrc.php mkdir /var/www/.drush cp /usr/local/src/drush-master/examples/example.drushrc.php /var/www/.drush/drushrc.php chown -R www-data:www-data /var/www/.drush
Edit both files to change:
// Specify a particular multisite. # $options['l'] = 'http://example.com/subdir'; $options['l'] = 'http://crin1.crin.org/'; // Specify your Drupal core base directory (useful if you use symlinks). # $options['r'] = '/home/USER/workspace/drupal-6'; $options['r'] = '/var/www/drupal';
Note this will need changing when the site is made live.
Test:
sudo -i su - www-data -s /bin/bash cd drupal/ drush uli Unable to load autoload.php. Drush now requires Composer in order to install its dependencies and autoload classes. Please see README.md
So, following https://getcomposer.org/doc/00-intro.md#globally
sudo -i cd /usr/local/src aptitude install curl curl -sS https://getcomposer.org/installer | php #!/usr/bin/env php All settings correct for using Composer Downloading... Composer successfully installed to: /usr/local/src/composer.phar Use it: php composer.phar mv composer.phar /usr/local/bin/composer
Try drush again:
sudo -i su - www-data -s /bin/bash drush --version Unable to load autoload.php. Drush now requires Composer in order to install its dependencies and autoload classes. Please see README.md
So:
sudo -i cd /usr/local/src/drush-master composer install Loading composer repositories with package information Installing dependencies (including require-dev) from lock file - Installing d11wtq/boris (v1.0.8) Downloading: Connecting... Failed to download d11wtq/boris from dist: The "https://api.github.com/repos/d11wtq/boris/zipball/125dd4e5752639af7678a22ea597115646d89c6e" file could not be downloaded (HTTP/1.1 404 Not Found) Now trying to download from source - Installing d11wtq/boris (v1.0.8) Cloning 125dd4e5752639af7678a22ea597115646d89c6e [RuntimeException] Failed to clone git@github.com:d11wtq/boris.git, git was not found, check that it is installed and in your PATH env. sh: 1: git: not found install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-plugins] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [--ignore-platform-reqs] [packages1] ... [packagesN] aptitude install git composer install Loading composer repositories with package information Installing dependencies (including require-dev) from lock file - Installing d11wtq/boris (v1.0.8) Downloading: Connecting... Failed to download d11wtq/boris from dist: The "https://api.github.com/repos/d11wtq/boris/zipball/125dd4e5752639af7678a22ea597115646d89c6e" file could not be downloaded (HTTP/1.1 404 Not Found) Now trying to download from source - Installing d11wtq/boris (v1.0.8) Cloning 125dd4e5752639af7678a22ea597115646d89c6e - Installing pear/console_table (1.2.1) Downloading: 100% - Installing symfony/var-dumper (v2.6.3) Downloading: 100% - Installing phpunit/php-token-stream (1.4.0) Downloading: 100% - Installing symfony/yaml (v2.6.3) Downloading: 100% - Installing sebastian/version (1.0.4) Downloading: 100% - Installing sebastian/recursion-context (1.0.0) Downloading: 100% - Installing sebastian/global-state (1.0.0) Downloading: 100% - Installing sebastian/exporter (1.2.0) Downloading: 100% - Installing sebastian/environment (1.2.1) Downloading: 100% - Installing sebastian/diff (1.2.0) Downloading: 100% - Installing sebastian/comparator (1.1.1) Downloading: 100% - Installing phpunit/php-text-template (1.2.0) Downloading: 100% - Installing doctrine/instantiator (1.0.4) Downloading: 100% - Installing phpunit/phpunit-mock-objects (2.3.0) Downloading: 100% - Installing phpunit/php-timer (1.0.5) Downloading: 100% - Installing phpunit/php-file-iterator (1.3.4) Downloading: 100% - Installing phpunit/php-code-coverage (2.0.15) Downloading: 100% - Installing phpunit/phpunit (4.4.5) Downloading: 100% - Installing symfony/process (v2.4.5) Downloading: 100% pear/console_table suggests installing pear/Console_Color2 (>=0.1.2) symfony/var-dumper suggests installing ext-symfony_debug () sebastian/global-state suggests installing ext-uopz (*) phpunit/php-code-coverage suggests installing ext-xdebug (>=2.2.1) phpunit/phpunit suggests installing phpunit/php-invoker (~1.1) Generating autoload files
And now Drush runs:
drush --version Drush Version : 7.0-dev drush status Drupal version : 7.24 Site URI : http://crin1.crin.org/ Database driver : mysql Database hostname : crin1 Database port : Database username : drupal Database name : drupal PHP executable : /usr/bin/php PHP configuration : /etc/php5/cli/php.ini PHP OS : Linux Drush script : /usr/local/src/drush-master/drush.php Drush version : 7.0-dev Drush temp directory : /tmp Drush configuration : /var/www/.drush/drushrc.php Drush alias files : Drupal root : /var/www/drupal Site path : sites/default
But we need to set up memcache:
drush uli PHP Fatal error: Class 'MemCacheDrupal' not found in /var/www/drupal/includes/cache.inc on line 31 Drush command terminated abnormally due to an unrecoverable error. [error] Error: Class 'MemCacheDrupal' not found in /var/www/drupal/includes/cache.inc, line 31
comment:8 in reply to: ↑ 7 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 1.61
- Total Hours changed from 2.9 to 4.51
Replying to chris:
But we need to set up memcache:
drush uli PHP Fatal error: Class 'MemCacheDrupal' not found in /var/www/drupal/includes/cache.inc on line 31 Drush command terminated abnormally due to an unrecoverable error. [error] Error: Class 'MemCacheDrupal' not found in /var/www/drupal/includes/cache.inc, line 31
This bug says:
In fact, there is a way to specify memcache cache handler in settings.php, using $conf['cache_backends'] variable, like this: $conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc';
We have:
$conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc'; // The 'cache_form' bin must be assigned no non-volatile storage. $conf['cache_class_cache_form'] = 'DrupalDatabaseCache'; $conf['cache_default_class'] = 'MemCacheDrupal'; $conf['memcache_key_prefix'] = 'crin_';
Following comment #29, I have changed it to:
include_once DRUPAL_ROOT . '/includes/cache.inc'; include_once DRUPAL_ROOT . '/sites/all/modules/memcache/memcache.inc'; $conf['cache_default_class'] = 'MemCacheDrupal';
And this seem to work, drush runs without a problem.
Checking https://crin2.crin.org/en/admin/reports/status there are these issues:
Required PHP extension not found. Install the memcache (recommended) or memcached extension.
So:
aptitude install php5-memcache /etc/init.d/php5-fpm restart
Other packages needed:
aptitude install php5-memcache php5-curl
We have this:
Multibyte string input conversion in PHP is active and must be disabled. Check the php.ini mbstring.http_input setting.
However according to the docs we should:
PHP 5.6 and later users should leave this empty and set default_charset instead.
And we have:
default_charset = "UTF-8"
On the live server we have no default charset set and:
mbstring.http_input = pass mbstring.http_output = pass
So these settings were copied to Crin2.
Setting up a crontab for the www-data user:
crontab -e -u www-data
58 * * * * /usr/bin/env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin COLUMNS=72 /usr/local/bin/drush --root=/var/www/drupal --uri=crin1.crin.org --quiet cron
TODO: Note the above will need updating when the site is made live.
We have:
GD library rotate and desaturate effects Not installed
Checking what the live server has:
aptitude search gd | grep ^i i A libgd2-xpm - GD Graphics Library version 2 i libgdbm3 - GNU dbm database routines (runtime version i A libgdk-pixbuf2.0-0 - GDK Pixbuf library i A libgdk-pixbuf2.0-common - GDK Pixbuf library - data files i php5-gd - GD module for php5
And Crin2 has:
aptitude search gd | grep ^i i A libgd3 - GD Graphics Library i libgdbm3 - GNU dbm database routines (runtime version i A libgdk-pixbuf2.0-0 - GDK Pixbuf library i A libgdk-pixbuf2.0-common - GDK Pixbuf library - data files
So:
aptitude install php5-gd service php5-fpm restart
Now the only outstanding issue on the status page is:
Apache Solr Your site was unable to contact the Apache Solr server.
Default environment url:
http://web1:8080/solr
On the live server we have:
aptitude search solr | grep ^i i A libsolr-java - Enterprise search server based on Lucene - i A solr-common - Enterprise search server based on Lucene3 i solr-tomcat - Enterprise search server based on Lucene3
So:
aptitude install libsolr-java solr-tomcat solr-common
Check the settings:
- https://crin2.crin.org/en/admin/config/search/apachesolr/settings
- https://crin2.crin.org/en/admin/config/search/apachesolr/search-pages
The "Solr server URL" was changed to: http://localhost:8080/solr.
TODO: this will need doing again after the final data sync, when the site on Crin2 is made live.
On the live server we have a password protected Nginx reverse proxy to the Solr admin interface, so the following was added to /etc/nginx/sites-available/solr.crin.org:
# default virtual server server { # listen for ipv4 # http://nginx.org/en/docs/http/ngx_http_core_module.html#listen listen 80; # server name and server aliases # http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name server_name solr.crin.org; # Prevent access to any files starting with a dot, like .htaccess # or text editor temp files location ~ /\. { access_log off; log_not_found off; deny all; } # Prevent access to tmp files created by vim location ~ .~$ { return 403; } location / { rewrite ^/(.*)$ https://solr.crin.org/$1? permanent; } } # HTTPS server # server { #listen 4430; listen 443; server_name solr.crin.org; access_log /var/log/nginx/solr.crin.org.ssl_access.log; error_log /var/log/nginx/solr.crin.org.ssl_error.log notice; ssl on; ssl_certificate /etc/ssl/cacert/crin2_cert.chained.pem; ssl_certificate_key /etc/ssl/cacert/crin2_privatekey.pem; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA; ssl_prefer_server_ciphers on; #add_header Strict-Transport-Security max-age=31536000; # https://developer.mozilla.org/en-US/docs/HTTP/X-Frame-Options add_header X-Frame-Options SAMEORIGIN; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://127.0.0.1:8080/; satisfy any; deny all; auth_basic "Solr Admin"; auth_basic_user_file /var/www/.htpasswd; } }
And the site was enabled and tested:
cd /etc/nginx/sites-enabled ln -s ../sites-available/solr.crin.org 20-solr.crin.org service nginx configtest [ ok ] Testing nginx configuration:. service nginx restart
And a sub-domain was created at a DNS level.
Create a username / passwd for the Solr admin interface:
cd /var/www/ aptitude install apache2-utils htpasswd -c .htpasswd crin
So, now the site is fully ready for testing: https://crin2.crin.org/
comment:9 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.1
- Total Hours changed from 4.51 to 4.61
Current status of the Drupal code in terms of available updates:
sudo -i su - www-data -s /bin/bash cd drupal/ drush status Drupal version : 7.24 Site URI : http://crin1.crin.org/ Database driver : mysql Database hostname : crin1 Database port : Database username : drupal Database name : drupal PHP executable : /usr/bin/php PHP configuration : /etc/php5/cli/php.ini PHP OS : Linux Drush script : /usr/local/src/drush-master/drush.php Drush version : 7.0-dev Drush temp directory : /tmp Drush configuration : /var/www/.drush/drushrc.php Drush alias files : Drupal root : /var/www/drupal Site path : sites/default drush up -n | grep available Drupal 7.24 7.37 SECURITY UPDATE available Views Bulk Operations (views_bulk_operations) 7.x-3.1 7.x-3.2 Update available Administration menu (admin_menu) 7.x-3.0-rc4 7.x-3.0-rc5 Update available Apache Solr framework (apachesolr) 7.x-1.6 7.x-1.7 Update available Apache Solr Multisite Search (apachesolr_multisitesearch) 7.x-1.0 7.x-1.1 Update available Autocomplete Deluxe (autocomplete_deluxe) 7.x-2.0-beta3 7.x-2.1 Update available Better Exposed Filters (better_exposed_filters) 7.x-3.0-beta3 7.x-3.2 Update available Chaos tools (ctools) 7.x-1.3 7.x-1.7 SECURITY UPDATE available CAPTCHA (captcha) 7.x-1.1 7.x-1.3 Update available CKEditor (ckeditor) 7.x-1.13 7.x-1.16 SECURITY UPDATE available Facet API (facetapi) 7.x-1.3 7.x-1.5 Update available Date (date) 7.x-2.6 7.x-2.8 SECURITY UPDATE available Devel (devel) 7.x-1.3 7.x-1.5 Update available Domain Access (domain) 7.x-3.10 7.x-3.11 Update available Entity API (entity) 7.x-1.2 7.x-1.6 SECURITY UPDATE available Entityforms (entityform) 7.x-2.0-beta4 7.x-2.0-rc1 Update available Field collection (field_collection) 7.x-1.0-beta5 7.x-1.0-beta8 Update available Fieldgroup (field_group) 7.x-1.3 7.x-1.4 Update available Google Map Field (google_map_field) 7.x-2.4 7.x-2.13 Update available Internationalization (i18n) 7.x-1.10 7.x-1.13 SECURITY UPDATE available IMCE (imce) 7.x-1.7 7.x-1.9 Update available jQuery Update (jquery_update) 7.x-2.3 7.x-2.5 Update available Localization update (l10n_update) 7.x-1.0-beta3 7.x-1.1 Update available Libraries (libraries) 7.x-2.1 7.x-2.2 Update available LoginToboggan (logintoboggan) 7.x-1.3 7.x-1.5 SECURITY UPDATE available Memcache (memcache) 7.x-1.2 7.x-1.5 Update available Menu attributes (menu_attributes) 7.x-1.0-rc2 7.x-1.0-rc3 Update available Menu Block (menu_block) 7.x-2.3 7.x-2.5 Update available Menu Node Views (menu_node_views) 7.x-1.x-dev 7.x-1.x-dev Update available Mime Mail (mimemail) 7.x-1.0-beta1 7.x-1.0-beta3 SECURITY UPDATE available MultiBlock (multiblock) 7.x-1.1 7.x-1.2 Update available OAuth (oauth) 7.x-3.1 7.x-3.2 Update available reCAPTCHA (recaptcha) 7.x-1.11 7.x-1.12 Update available Rules (rules) 7.x-2.6 7.x-2.9 Update available Search API (search_api) 7.x-1.6 7.x-1.14 Update available Database search (search_api_db) 7.x-1.2 7.x-1.4 Update available Taxonomy menu (taxonomy_menu) 7.x-1.4 7.x-1.5 Update available Token (token) 7.x-1.5 7.x-1.6 Update available Transliteration (transliteration) 7.x-3.1 7.x-3.2 Update available Variable (variable) 7.x-2.3 7.x-2.5 Update available Views (views) 7.x-3.7 7.x-3.11 SECURITY UPDATE available Views Autocomplete Filters (views_autocomplete_filters) 7.x-1.0 7.x-1.2 Update available NOTE: A security update for the Drupal core is available. Cancelled. [cancel] Multibyte string input conversion in PHP is active and must be disabled. Check the php.ini mbstring.http_input setting. Please refer to the[warning] PHP mbstring documentation for more information. (Currently using Unicode library Error) No database updates required [success] 'all' cache was cleared. [success] Finished performing updates. [ok]
So it looks like there is perhaps still an issue with "Multibyte string input conversion in PHP".
Jonas, the security updates listed above might contain issues which apply to CRIN, can check about these with the web developers?
comment:10 follow-up: ↓ 12 Changed 3 years ago by jonas
I tested crin2.crin.org and the site is a bit slow. I was able to log in to the CMS, but apart from that I haven't done much.
comment:11 follow-up: ↓ 13 Changed 3 years ago by jonas
The problem is that currently, we don't have any web developers, only people who are assessing the work that Effusion did. However, they work on a tight deadline and won't have time to check this, also it's not what my colleagues want them to do.
So I think we will have to wait until we have new web developer, but then it will be too late, right?
comment:12 in reply to: ↑ 10 Changed 3 years ago by chris
Replying to jonas:
I tested crin2.crin.org and the site is a bit slow.
OK, I'll see what can be done to speed things up by adjusting the MySQL / PHP-FPM and Nginx memory usage etc.
I was able to log in to the CMS, but apart from that I haven't done much.
Could you test uploading a file?
comment:13 in reply to: ↑ 11 Changed 3 years ago by chris
Replying to jonas:
So I think we will have to wait until we have new web developer, but then it will be too late, right?
I don't know if any of the security issues listed in ticket:6#comment:9 apply to your site, if you would like I could spend some time reading up on them to see if it is clear if any do, however I wouldn't feel confident that I could do any Drupal updates without the potential for breaking things (this is why there are lot of Drupal sites in situations like this -- updating things often breaks them).
Changed 3 years ago by chris
comment:14 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.28
- Total Hours changed from 4.61 to 4.89
Testing using pingdom.com we have:
- https://crin2.crin.org/ - Load time: 3.42s
- https://www.crin.org/ - Load time: 2.36s
So, lets see what we MySQL settings can be changed, on Crin1:
aptitude install mysqltuner mysqltuner >> MySQLTuner 1.3.0 - Major Hayden <major@mhtx.net> >> Bug reports, feature requests, and downloads at http://mysqltuner.com/ >> Run with '--help' for additional options and output filtering [OK] Logged in using credentials from debian maintenance account. [OK] Currently running supported MySQL version 5.5.43-0+deb8u1 [OK] Operating on 64-bit architecture -------- Storage Engine Statistics ------------------------------------------- [--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MRG_MYISAM [--] Data in InnoDB tables: 718M (Tables: 506) [--] Data in MEMORY tables: 0B (Tables: 1) [--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17) [--] Data in MyISAM tables: 953M (Tables: 79) [!!] Total fragmented tables: 518 -------- Security Recommendations ------------------------------------------- [OK] All database users have passwords assigned -------- Performance Metrics ------------------------------------------------- [--] Up for: 5d 1h 0m 15s (133K q [0.306 qps], 1K conn, TX: 5B, RX: 172M) [--] Reads / Writes: 41% / 59% [--] Total buffers: 192.0M global + 2.7M per thread (151 max threads) [OK] Maximum possible memory usage: 597.8M (15% of installed RAM) [OK] Slow queries: 0% (16/133K) [OK] Highest usage of available connections: 2% (4/151) [OK] Key buffer size / total MyISAM indexes: 16.0M/238.9M [OK] Key buffer hit rate: 96.9% (24M cached / 757K reads) [OK] Query cache efficiency: 73.5% (67K cached / 91K selects) [!!] Query cache prunes per day: 2228 [OK] Sorts requiring temporary tables: 0% (3 temp sorts / 2K sorts) [OK] Temporary tables created on disk: 10% (74 on disk / 713 total) [OK] Thread cache hit rate: 99% (4 created / 1K connections) [!!] Table cache hit rate: 5% (400 open / 7K opened) [OK] Open file limit used: 12% (128/1K) [OK] Table locks acquired immediately: 99% (63K immediate / 63K locks) [!!] InnoDB buffer pool / data size: 128.0M/718.5M [OK] InnoDB log waits: 0 -------- Recommendations ----------------------------------------------------- General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance Enable the slow query log to troubleshoot bad queries Increase table_open_cache gradually to avoid file descriptor limits Read this before increasing table_open_cache over 64: http://bit.ly/1mi7c4C Variables to adjust: query_cache_size (> 16M) table_open_cache (> 400) innodb_buffer_pool_size (>= 718M)
Based on the above edit /etc/mysql/my.cnf and change:
#query_cache_size = 16M query_cache_size = 32M innodb_buffer_pool_size = 1024M
Restart MySQL:
service mysql restart
Now the results of mysqltuner can be checked again tomorrow (the server needs to be running a while to generate stats).
For future reference the existing db1.crin.org /etc/mysql/my.cnf has been attached, I'm not simply copying this over since the existing db1.crin.org GreenQloud server has more resources available to it -- it doesn't host web sites.
Note that db1.crin.org uses a RAM disk for the tmpdir.
Changed 3 years ago by chris
comment:15 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.25
- Total Hours changed from 4.89 to 5.14
Attached is the /etc/php5/fpm/php.ini from the existing live GreenQloud server, web1.crin.org:
Comparing the configurations these things were changed in /etc/php5/fpm/php.ini on Crin2:
;post_max_size = 8M post_max_size = 20M ;upload_max_filesize = 2M upload_max_filesize = 20M ;max_file_uploads = 20 max_file_uploads = 60 extension=uploadprogress.so
Following Install PECL uploadprogress on Debian 7 Wheezy:
aptitude install make php5-dev php-pear pecl install uploadprogress service php5-fpm restart
comment:16 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.25
- Total Hours changed from 5.14 to 5.39
To enable spdy lines like this were changed in the Nginx config:
listen 443 ssl spdy default_server;
And this was tested at http://spdycheck.org/
Note that the spdycheck.org site incorrectly resported that HSTS isn't in use, but is is, see the SSLLabs report:
This seems to have helped with the speed of the site -- it now loads in less than 2.5 seconds via the Pingdom full page test from Sweden and less than 2 seconds from Amsterdam.
Also memcached was added to the Munin memory graph -- currently it is set to use 64M of RAM, this could be increased if needs be.
Jonas -- does the site seem quicker now? Do file uploads report their progress (I'm wondering if I should have installed a Nginx specific version of upload progress)?
I noticed that Google Analytics, as well a Piwik, is enabled on this site.
comment:17 Changed 3 years ago by chris
Benchmarking the front page from another server at 1984.is, requesting the front page 10k times with 20 concurrent requests:
ab -n 10000 -c20 http://crin2.crin.org/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking crin2.crin.org (be patient) Completed 1000 requests Completed 2000 requests Completed 3000 requests Completed 4000 requests Completed 5000 requests Completed 6000 requests Completed 7000 requests Completed 8000 requests Completed 9000 requests Completed 10000 requests Finished 10000 requests Server Software: nginx/1.6.2 Server Hostname: crin2.crin.org Server Port: 80 Document Path: / Document Length: 74044 bytes Concurrency Level: 20 Time taken for tests: 1050.591 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 745820000 bytes HTML transferred: 740440000 bytes Requests per second: 9.52 [#/sec] (mean) Time per request: 2101.182 [ms] (mean) Time per request: 105.059 [ms] (mean, across all concurrent requests) Transfer rate: 693.27 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 7 53.1 2 1032 Processing: 596 2093 679.7 1937 5671 Waiting: 559 2066 672.3 1915 5637 Total: 598 2099 683.3 1942 5680 Percentage of the requests served within a certain time (ms) 50% 1942 66% 2264 75% 2474 80% 2616 90% 3012 95% 3447 98% 3963 99% 4242 100% 5680 (longest request)
So, Requests per second: 9.52 and Time per request: 2 sec, and looking at the Munin graphs for the servers they didn't really break a sweat, but the number of requests per second could probably be higher with some tweaks, another test:
ab -n 100 -c40 https://crin2.crin.org/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking crin2.crin.org (be patient).....done Server Software: nginx/1.6.2 Server Hostname: crin2.crin.org Server Port: 443 SSL/TLS Protocol: TLSv1/SSLv3,ECDHE-RSA-AES128-GCM-SHA256,2048,128 Document Path: / Document Length: 74087 bytes Concurrency Level: 40 Time taken for tests: 28.608 seconds Complete requests: 100 Failed requests: 0 Write errors: 0 Total transferred: 7467605 bytes HTML transferred: 7408700 bytes Requests per second: 3.50 [#/sec] (mean) Time per request: 11443.296 [ms] (mean) Time per request: 286.082 [ms] (mean, across all concurrent requests) Transfer rate: 254.91 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 15 91 81.8 46 260 Processing: 4439 6857 2554.0 6115 28492 Waiting: 4228 6581 1352.8 6061 10188 Total: 4535 6948 2586.8 6188 28606 Percentage of the requests served within a certain time (ms) 50% 6188 66% 7293 75% 7667 80% 7941 90% 9243 95% 9866 98% 10487 99% 28606 100% 28606 (longest request)
It's slower with HTTPS by the looks of this, without it:
ab -n 100 -c40 http://crin2.crin.org/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking crin2.crin.org (be patient).....done Server Software: nginx/1.6.2 Server Hostname: crin2.crin.org Server Port: 80 Document Path: / Document Length: 74044 bytes Concurrency Level: 40 Time taken for tests: 11.985 seconds Complete requests: 100 Failed requests: 0 Write errors: 0 Total transferred: 7458200 bytes HTML transferred: 7404400 bytes Requests per second: 8.34 [#/sec] (mean) Time per request: 4793.849 [ms] (mean) Time per request: 119.846 [ms] (mean, across all concurrent requests) Transfer rate: 607.73 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 1 14 99.4 4 998 Processing: 574 4007 1096.9 4473 5386 Waiting: 555 3983 1100.9 4451 5371 Total: 581 4021 1104.9 4474 5543 Percentage of the requests served within a certain time (ms) 50% 4474 66% 4614 75% 4692 80% 4738 90% 4925 95% 5133 98% 5390 99% 5543 100% 5543 (longest request)
8 requests per second compared with 3.5, quite a difference.
On the GreenQloud server in /etc/nginx/nginx.conf we have:
#worker_processes 4; worker_processes 127;
And on Crin2 we have:
worker_processes 4;
Doubling this and testing what it does for the benchmark:
ab -n 100 -c40 http://crin2.crin.org/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking crin2.crin.org (be patient).....done Server Software: nginx/1.6.2 Server Hostname: crin2.crin.org Server Port: 80 Document Path: / Document Length: 74044 bytes Concurrency Level: 40 Time taken for tests: 7.715 seconds Complete requests: 100 Failed requests: 0 Write errors: 0 Total transferred: 7458200 bytes HTML transferred: 7404400 bytes Requests per second: 12.96 [#/sec] (mean) Time per request: 3085.845 [ms] (mean) Time per request: 77.146 [ms] (mean, across all concurrent requests) Transfer rate: 944.10 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 2 1.7 2 6 Processing: 277 2468 820.1 2795 3431 Waiting: 268 2455 819.0 2788 3417 Total: 279 2471 819.5 2800 3432 Percentage of the requests served within a certain time (ms) 50% 2800 66% 2923 75% 3011 80% 3043 90% 3159 95% 3249 98% 3386 99% 3432 100% 3432 (longest request)
12 requests per second rather than 8 and no fails, doubling the number of Nginx processes again and testing:
ab -n 100 -c40 http://crin2.crin.org/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking crin2.crin.org (be patient).....done Server Software: nginx/1.6.2 Server Hostname: crin2.crin.org Server Port: 80 Document Path: / Document Length: 74044 bytes Concurrency Level: 40 Time taken for tests: 9.192 seconds Complete requests: 100 Failed requests: 0 Write errors: 0 Total transferred: 7458200 bytes HTML transferred: 7404400 bytes Requests per second: 10.88 [#/sec] (mean) Time per request: 3676.632 [ms] (mean) Time per request: 91.916 [ms] (mean, across all concurrent requests) Transfer rate: 792.40 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 1 3 3.4 2 26 Processing: 336 3150 995.5 3499 4361 Waiting: 322 3131 1000.3 3488 4344 Total: 338 3153 996.2 3500 4365 Percentage of the requests served within a certain time (ms) 50% 3500 66% 3746 75% 3874 80% 3910 90% 4056 95% 4196 98% 4269 99% 4365 100% 4365 (longest request)
Slower, so setting the worker_processes to 8 for now.
In /etc/php5/fpm/pool.d/www.conf we have:
pm.max_children = 5
And on the live GreenQloud server we have:
pm.max_children = 127
Another test :
crin2.crin.org (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: nginx/1.6.2 Server Hostname: crin2.crin.org Server Port: 80 Document Path: / Document Length: 74044 bytes Concurrency Level: 80 Time taken for tests: 123.974 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 74582000 bytes HTML transferred: 74044000 bytes Requests per second: 8.07 [#/sec] (mean) Time per request: 9917.929 [ms] (mean) Time per request: 123.974 [ms] (mean, across all concurrent requests) Transfer rate: 587.49 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 6 31.8 3 998 Processing: 506 9654 2600.3 9653 16149 Waiting: 497 9627 2594.4 9629 16108 Total: 510 9659 2601.9 9659 16152 Percentage of the requests served within a certain time (ms) 50% 9659 66% 10598 75% 11121 80% 11789 90% 12971 95% 14076 98% 15243 99% 15664 100% 16152 (longest request)
8 requests per second, so this isn't hlping, it seems, testing with the live server:
www.crin.org (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: nginx/1.2.1 Server Hostname: www.crin.org Server Port: 80 Document Path: / Document Length: 74026 bytes Concurrency Level: 80 Time taken for tests: 15.556 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 74599000 bytes HTML transferred: 74026000 bytes Requests per second: 64.28 [#/sec] (mean) Time per request: 1244.470 [ms] (mean) Time per request: 15.556 [ms] (mean, across all concurrent requests) Transfer rate: 4683.15 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 40 41 2.5 41 58 Processing: 298 1170 166.2 1180 1796 Waiting: 172 1045 165.7 1056 1674 Total: 339 1212 166.1 1221 1847 Percentage of the requests served within a certain time (ms) 50% 1221 66% 1264 75% 1292 80% 1305 90% 1376 95% 1458 98% 1565 99% 1665 100% 1847 (longest request)
64 requests per second, something must be missing? With HTTPS:
ab -n 1000 -c80 https://www.crin.org/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking www.crin.org (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: nginx/1.2.1 Server Hostname: www.crin.org Server Port: 443 SSL/TLS Protocol: TLSv1/SSLv3,ECDHE-RSA-AES128-GCM-SHA256,2048,128 Document Path: / Document Length: 74069 bytes Concurrency Level: 80 Time taken for tests: 15.981 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 74693000 bytes HTML transferred: 74069000 bytes Requests per second: 62.57 [#/sec] (mean) Time per request: 1278.502 [ms] (mean) Time per request: 15.981 [ms] (mean, across all concurrent requests) Transfer rate: 4564.24 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 165 222 97.3 191 594 Processing: 305 1024 167.6 1017 1605 Waiting: 237 912 162.9 899 1485 Total: 632 1246 182.6 1229 2199 Percentage of the requests served within a certain time (ms) 50% 1229 66% 1293 75% 1321 80% 1343 90% 1415 95% 1536 98% 1868 99% 2067 100% 2199 (longest request)
16 requests per second, the 1984.is servers:
ab -n 1000 -c80 https://crin2.crin.org/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking crin2.crin.org (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: nginx/1.6.2 Server Hostname: crin2.crin.org Server Port: 443 SSL/TLS Protocol: TLSv1/SSLv3,ECDHE-RSA-AES128-GCM-SHA256,2048,128 Document Path: / Document Length: 74087 bytes Concurrency Level: 80 Time taken for tests: 114.247 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 74676000 bytes HTML transferred: 74087000 bytes Requests per second: 8.75 [#/sec] (mean) Time per request: 9139.735 [ms] (mean) Time per request: 114.247 [ms] (mean, across all concurrent requests) Transfer rate: 638.32 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 11 75 190.4 27 1274 Processing: 535 8487 2552.3 7933 15406 Waiting: 516 8465 2548.7 7917 15386 Total: 650 8562 2548.7 7960 15911 Percentage of the requests served within a certain time (ms) 50% 7960 66% 8800 75% 9588 80% 10074 90% 12444 95% 14361 98% 14994 99% 15243 100% 15911 (longest request)
9 requests per second, close but slower. Looking at the spikes on the Munin graphs for both servers there is still a huge amount of unused CPU and RAM so with tweaking these figures should be able to be improved.
comment:18 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.25
- Total Hours changed from 5.39 to 5.64
Email was configured on Crin2, the following was added to /etc/aliases:
root: chris@webarchitects.co.uk,admin@crin.org
And newaliases was run and also dpkg-reconfigure exim4-config and logwatch and metche were installed and email was tested by sending email to root.
In addition both server have been rebooted due to venom.
comment:19 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.75
- Total Hours changed from 5.64 to 6.39
The current live GreenQloud server has APC installed:
aptitude search apc | grep ^i i php-apc - APC (Alternative PHP Cache) module for PHP
This could be a cause for the slowness of the Drupal site, so on Crin2:
aptitude search apc | grep php p php-apc - APC User Cache for PHP 5 (transitional pac p php5-apcu - APC User Cache for PHP 5 aptitude install php5-apcu logchange "php5-apcu : installed"
On the GreenQloud server /etc/php5/mods-available/apc.ini contains:
extension=apc.so apc.shm_size="512M" apc.max_file_size="2M"
And we might want to add these settings, but to know what they can best be set to we need some Munin stats, so this munin-php-apc plugin looks like it will do the job, so:
cd /usr/local/share/munin/plugins/ wget https://github.com/geerlingguy/munin-php-apc/archive/master.zip unzip master.zip Archive: master.zip bc7f1ad0103bbf500e8d36e342cfaa53733fbc7d creating: munin-php-apc-master/ inflating: munin-php-apc-master/CHANGELOG.txt inflating: munin-php-apc-master/README.md inflating: munin-php-apc-master/apc_info.php inflating: munin-php-apc-master/php_apc_
Make a directory and move the apc_info.php file:
mkdir /var/www/localhost mv munin-php-apc-master/apc_info.php /var/www/localhost/apc-info.php
Edit /etc/nginx/sites-available/localhost to add:
root /var/www/localhost; location ~ /apc-info\.php$ { fastcgi_pass unix:/var/run/php5-fpm.sock; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_intercept_errors on; include fastcgi_params; access_log off; allow 127.0.0.1; deny all; }
Test and restart and test:
service nginx configtest [ ok ] Testing nginx configuration:. service nginx restart lynx -dump http://localhost/apc-info.php size: 33554296 used: 32992 free: 33521304 hits: 0.00 misses: 0.00 request_rate: 0.00 hit_rate: 0.00 miss_rate: 0.00 insert_rate: 0.00 entries: 0 inserts: 0 purges: purge_rate: 100.00 fragment_percentage: 0.00 fragmented: 0.00 fragment_segments: 0 optcode_size: 0 user_size: 0 user_hits: 0.00 user_misses: 0.00 user_request_rate: 0.00 user_hit_rate: 0.00 user_miss_rate: 0.00 user_insert_rate: 0.00 user_entries: 0 user_inserts: 0 user_purges: user_purge_rate: 0.00
So now to enable the munin plugin:
cd /etc/munin/plugins/ ln -s /usr/local/share/munin/plugins/munin-php-apc-master/php_apc_ php_apc_files ln -s /usr/local/share/munin/plugins/munin-php-apc-master/php_apc_ php_apc_fragmentation ln -s /usr/local/share/munin/plugins/munin-php-apc-master/php_apc_ php_apc_hit_miss ln -s /usr/local/share/munin/plugins/munin-php-apc-master/php_apc_ php_apc_purge ln -s /usr/local/share/munin/plugins/munin-php-apc-master/php_apc_ php_apc_rates ln -s /usr/local/share/munin/plugins/munin-php-apc-master/php_apc_ php_apc_usage ln -s /usr/local/share/munin/plugins/munin-php-apc-master/php_apc_ php_apc_mem_size ln -s /usr/local/share/munin/plugins/munin-php-apc-master/php_apc_ php_apc_user_hit_miss ln -s /usr/local/share/munin/plugins/munin-php-apc-master/php_apc_ php_apc_user_entries ln -s /usr/local/share/munin/plugins/munin-php-apc-master/php_apc_ php_apc_user_rates
Add the following to /etc/munin/plugin-conf.d/munin-node:
[php_apc_*] user root env.url http://localhost/apc-info.php?auto
Test and restart:
cd /etc/munin/plugins/ munin-run php_apc_files used.value 32992 free.value 33521304 hits.value 0.00 misses.value 0.00 request_rate.value 0.00 hit_rate.value 0.00 miss_rate.value 0.00 insert_rate.value 0.00 entries.value 0 inserts.value 0 purges.value U purge_rate.value 100.00 fragmented.value 0.00 fragment_segments.value 0 fragment_percentage.value 0.00 optcode_size.value 0 user_size.value 0 user_hits.value 0.00 user_misses.value 0.00 user_request_rate.value 0.00 user_hit_rate.value 0.00 user_miss_rate.value 0.00 user_insert_rate.value 0.00 user_entries.value 0 user_inserts.value 0 user_purges.value U user_purge_rate.value 0.00 service munin-node restart
And we should soon have some APC stats here:
Testing to see if there is noticable speed increase:
ab -n 1000 -c80 https://crin2.crin.org/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking crin2.crin.org (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: nginx/1.6.2 Server Hostname: crin2.crin.org Server Port: 443 SSL/TLS Protocol: TLSv1/SSLv3,ECDHE-RSA-AES128-GCM-SHA256,2048,128 Document Path: / Document Length: 74087 bytes Concurrency Level: 80 Time taken for tests: 67.240 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 74676000 bytes HTML transferred: 74087000 bytes Requests per second: 14.87 [#/sec] (mean) Time per request: 5379.193 [ms] (mean) Time per request: 67.240 [ms] (mean, across all concurrent requests) Transfer rate: 1084.56 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 8 35 78.0 12 1012 Processing: 161 5141 874.1 5278 6769 Waiting: 156 5136 874.0 5270 6766 Total: 305 5177 820.7 5293 6782 Percentage of the requests served within a certain time (ms) 50% 5293 66% 5392 75% 5487 80% 5556 90% 5708 95% 5822 98% 5932 99% 6014 100% 6782 (longest request)
15 requests per second, that is an improvement on the 9 requests per second we had with the same test last night.
Changed 3 years ago by chris
comment:20 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.1
- Total Hours changed from 6.39 to 6.49
The front page of the site loads in less than 2 seconds when tested via http://tools.pingdom.com/fpt using the Amsterdam test server.
comment:21 Changed 3 years ago by chris
Looking at the php-apc stats it doesn't appear to be doing anything, the config file is /etc/php5/fpm/conf.d/20-apcu.ini and it contains:
extension=apcu.so
The apc config documentation lists all the config options, to start with these were added:
apc.enabled=1 apc.shm_size=128M apc.max_file_size=1M
And service php5-fpm restart was run, and some more benchmarking was done using ab:
ab -n 1000 -c80 https://crin2.crin.org/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking crin2.crin.org (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: nginx/1.6.2 Server Hostname: crin2.crin.org Server Port: 443 SSL/TLS Protocol: TLSv1/SSLv3,ECDHE-RSA-AES128-GCM-SHA256,2048,128 Document Path: / Document Length: 74087 bytes Concurrency Level: 80 Time taken for tests: 57.527 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 74676000 bytes HTML transferred: 74087000 bytes Requests per second: 17.38 [#/sec] (mean) Time per request: 4602.125 [ms] (mean) Time per request: 57.527 [ms] (mean, across all concurrent requests) Transfer rate: 1267.69 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 8 33 76.1 12 1006 Processing: 312 4397 683.8 4459 6235 Waiting: 307 4390 683.6 4454 6222 Total: 432 4430 664.3 4479 6346 Percentage of the requests served within a certain time (ms) 50% 4479 66% 4625 75% 4739 80% 4807 90% 5045 95% 5209 98% 5369 99% 5555 100% 6346 (longest request)
17 pages a second and another Pingdom test came in at 1.85s for the front page.
comment:22 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.25
- Total Hours changed from 6.49 to 6.74
comment:23 Changed 3 years ago by chris
The MySQL sync has run but the sync of the files is still running.
comment:24 Changed 3 years ago by chris
Copying the files is taking a long time, a 214M file is taking 30 mins to copy...
comment:25 Changed 3 years ago by chris
The files have finally synced, but I'm going to be away from a computer between 3pm and 4pm so I don't want to do the DNS update right now, I'll do it at 4pm. Sorry for the delay, I wasn't expecting it to take so long to sync the files that have changed over the last few weeks.
comment:26 Changed 3 years ago by chris
I'm going to update the DNS now.
comment:27 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.2
- Total Hours changed from 6.74 to 6.94
I have changed these lines in the DNS for crin.org:
@ 900 IN A 93.95.228.180 enoc 900 IN A 93.95.228.180 www 900 IN A 93.95.228.180
And these lines for crin.com:
@ 900 IN A 93.95.228.180 www 900 IN A 93.95.228.180
Checking the DNS servers:
dig @NS0.1984.IS crin.org +short 46.149.19.215 dig @NS0.1984.IS crin.com +short 46.149.19.215
These DNS servers should update soon and then others will follow.
comment:28 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.4
- Total Hours changed from 6.94 to 7.34
The enoc.crin.org Nginx config file was copied from crin-web1.
The 1984.is DNS servers have updated:
dig @NS0.1984.IS crin.com +short 93.95.228.180 dig @NS0.1984.IS crin.org +short 93.95.228.180 dig @NS0.1984.IS enoc.crin.org +short 93.95.228.180
The main site at https://www.crin.org/ seems fine, the SSL settings get a A+ rating:
There is an issue with http://enoc.crin.org/ - it displays this message:
You are not authorized to access this page.
But the GreenQloud server does the same thing...
Everything seems to be working OK, I'll check the Munin graphs in a while to see what is happening load wise:
comment:29 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.3
- Total Hours changed from 7.34 to 7.64
Update Drupal crontab as per TODO note above:
crontab -e -u www-data
It now contains:
58 * * * * /usr/bin/env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin COLUMNS=72 /usr/local/bin/drush --root=/var/www/drupal --uri=www.crin.org --quiet cron
Test it:
su - www-data -s /bin/bash /usr/local/bin/drush --root=/var/www/drupal --uri=www.crin.org --quiet cron -su: /usr/local/bin/drush: Permission denied
So:
chmod 755 /usr/local/src/drush-master/drush
And try again:
/usr/local/bin/drush --root=/var/www/drupal --uri=www.crin.org --quiet cron Command core-cron needs a higher bootstrap level to run - you will need to invoke drush from a more functional Drupal environment [error] to run this command. The drush command 'cron' could not be executed. [error] Drush was not able to start (bootstrap) the Drupal database. [error] Hint: This may occur when Drush is trying to: * bootstrap a site that has not been installed or does not have a configured database. In this case you can select another site with a working database setup by specifying the URI to use with the --uri parameter on the command line. See `drush topic docs-aliases` for details. * connect the database through a socket. The socket file may be wrong or the php-cli may have no access to it in a jailed shell. See http://drupal.org/node/1428638 for details. Drush was attempting to connect to: Drupal version : 7.24 Site URI : www.crin.org Database driver : mysql Database hostname : crin1 Database port : Database username : drupal Database name : drupal PHP executable : /usr/bin/php PHP configuration : /etc/php5/cli/php.ini PHP OS : Linux Drush script : /usr/local/src/drush-master/drush.php Drush version : 7.0-dev Drush temp directory : /tmp Drush configuration : /var/www/.drush/drushrc.php Drush alias files : Drupal root : /var/www/drupal Site path : sites/default
Testing on the GreenQloud server:
su - www-data -s /bin/bash /usr/bin/env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin COLUMNS=72 /usr/local/sbin/drush --root=/var/www/crin --uri=www.crin.org --quiet cron PHP Fatal error: Class 'MemCacheDrupal' not found in /var/www/crin/includes/cache.inc on line 31 Drush command terminated abnormally due to an unrecoverable [error] error. Error: Class 'MemCacheDrupal' not found in /var/www/crin/includes/cache.inc, line 31
So it hasn't been working on the old server either, I have created a new ticket for this, ticket:18.
comment:30 Changed 3 years ago by chris
- Cc jenny gillian added; jonas removed
- Resolution set to fixed
- Status changed from new to closed
Closing this ticket now -- the main Drupal site, https://www.crin.org/ is now running on the 1984.is servers, follow up can be done on ticket:9.
comment:31 Changed 3 years ago by chris
- Add Hours to Ticket changed from 0 to 0.15
- Total Hours changed from 7.64 to 7.79
Jenny asked:
Are ok to upload info on the site now?
Yes you should be, the GreenCloud servers are still seeing some traffic so not everybodies DNS has updated yet, though all the recent requests are from robots, eg:
Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/ Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
The Googlebot doesn't appear to be a fake Googlebot, it has reverse DNS set up:
dig -x 66.249.64.189 +short crawl-66-249-64-189.googlebot.com.
I think best leave the old servers running for a day or two.
Adding ssh keys and doing an initial sync of the site, on crin2, generate a key pair:
Add the public key to web1 and set it to allow allow connections from web1, ~/.ssh/authorized_keys:
Create ~/.ssh/config containing:
Create a directory for the site on crin1, install and rsync the site (checking the ssh fingerprint):
The site is 20G so it'll take some time to copy:
Jonas: are these things needed on the new server?