CRIN Trac: Ticket #1: Trac initial install and configuration

status changed

chris — Tue, 28 Apr 2015 11:44:02 GMT

status changed from new to accepted

chris — Tue, 28 Apr 2015 12:00:06 GMT

Following the docs at:

Install packages:

aptitude install trac trac-email2trac apache2-mpm-itk libapache2-mod-wsgi

Create directory for trac:

mkdir /var/www/trac

Edit /root/.profile and /var/www/trac/.profile and add, see, see http://trac.edgewall.org/wiki/0.12/TracInstall#RunningtheStandaloneServer

export PKG_RESOURCES_CACHE_ZIP_MANIFESTS=1

Add trac user:

adduser --system --disabled-password --disabled-login --group --home=/var/www/trac trac
chown -R trac:trac /var/www/trac

Change to the trac user and run things as that user:

su - trac -s /bin/bash

Set up the env and create a passwd file and grant perms:

trac-admin /var/www/trac initenv
mkdir /var/www/trac/apache
trac-admin /var/www/trac deploy /var/www/trac/apache
htdigest -c /var/www/trac/.htpasswd trac chris
trac-admin /var/www/trac permission add chris admin
trac-admin /var/www/trac permission add chris TRAC_ADMIN

Enable some apache modules:

a2enmod ssl auth_digest

Create a apache config file at /etc/apache2/sites-available/trac.conf:

<VirtualHost trac.crin.org:80>
        <IfModule mpm_itk_module>
                AssignUserID trac trac
                MaxClientsVHost 60
        </IfModule>
        ServerName trac.crin.org
        ServerAdmin chris@webarchitects.co.uk
        Redirect / https://trac.crin.org/
</VirtualHost>
<IfModule mod_ssl.c>
        <VirtualHost trac.crin.org:443>
                <IfModule mpm_itk_module>
                        AssignUserID trac trac
                        MaxClientsVHost 60
                </IfModule>
                ServerName trac.crin.org
                ServerAdmin chris@webarchitects.co.uk
                # DocumentRoot /var/www/html
                #LogLevel info ssl:warn
                ErrorLog ${APACHE_LOG_DIR}/trac.error.log
                CustomLog ${APACHE_LOG_DIR}/trac.access.log combined
                SSLEngine on
                SSLCertificateFile      /etc/ssl/gandi/trac.crt.pem
                SSLCertificateKeyFile /etc/ssl/gandi/trac.key.pem
                SSLCACertificateFile /etc/ssl/gandi/root.pem
                # http://trac.edgewall.org/wiki/0.12/TracInstall#cgi-bin
                SetEnv PKG_RESOURCES_CACHE_ZIP_MANIFESTS 1
                Alias /trac/chrome/common /var/www/trac/apache/htdocs/common
                Alias /trac/chrome/site /var/www/trac/apache/htdocs/site
                <Directory "/var/www/trac/apache/htdocs">
                        Require all granted
                </Directory>
                WSGIScriptAlias /trac /var/www/trac/apache/cgi-bin/trac.wsgi
                <Directory "/var/www/trac/apache/cgi-bin/">
                        WSGIApplicationGroup %{GLOBAL}
                        SSLOptions +StdEnvVars
                        Require all granted
                </Directory>
                <Location "/trac/login">
                        AuthType Digest
                        AuthName "trac"
                        AuthDigestDomain /trac
                        AuthUserFile /var/www/trac/.htpasswd
                        Require valid-user
                </Location>
                RedirectMatch ^/$ https://trac.crin.org/trac
                BrowserMatch "MSIE [2-6]" \
                                nokeepalive ssl-unclean-shutdown \
                                downgrade-1.0 force-response-1.0
                BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
        </VirtualHost>
</IfModule>

Sort out TLS cert:

sudo -i
mkdir /etc/ssl/gandi
chmod 700 /etc/ssl/gandi
cd /etc/ssl/gandi
openssl req -nodes -newkey rsa:2048 -sha256 -keyout trac.key.pem -out trac.csr.pem
wget "https://www.gandi.net/static/CAs/GandiStandardSSLCA2.pem"
wget "http://crt.usertrust.com/USERTrustRSAAddTrustCA.crt"
openssl x509 -inform DER -in USERTrustRSAAddTrustCA.crt -out USERTrustRSAAddTrustCA.pem
cat USERTrustRSAAddTrustCA.pem > root.pem
cat GandiStandardSSLCA2.pem >> root.pem

chris — Tue, 28 Apr 2015 12:14:57 GMT

The above comment didn't result in a email, so...

Configured a mailserver:

dpkg-reconfigure exim4-config

Set these variables in /var/www/trac/conf/trac.ini:

[notification]
admit_domains =  trac.crin.org, crin1.crin.org
smtp_default_domain = trac.crin.org
smtp_from = trac@trac.crin.org
smtp_replyto = trac@trac.crin.org

Note that a MX record and a way to do incoming email hasn't been sorted out so the above will need changin when it is.

Testing to see if a email is now sent...

chris — Tue, 28 Apr 2015 12:26:42 GMT

That didn't work, from the /var/log/exim4/mainlog:

2015-04-28 12:14:57 1Yn4Ph-0002BL-61 ASPMX.L.GOOGLE.COM [2a00:1450:4013:c01::1a] Network is unreachable
2015-04-28 12:14:59 1Yn4Ph-0002BL-61 ** chris@webarchitects.co.uk R=dnslookup T=remote_smtp X=TLS1.2:RSA_AES_128_CBC_SHA1:128 DN="C=GB,ST=South Yorkshire,L=Sheffield,O=webarchitects.coop,CN=mx.webarch.net": SMTP error from remote mail server after RCPT TO:<chris@webarchitects.co.uk>: host mx.webarch.net [81.95.52.71]: 550 Invalid HELO
2015-04-28 12:14:59 1Yn4Ph-0002BL-61 ** jonas@crin.org R=dnslookup T=remote_smtp X=TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128 DN="C=US,ST=California,L=Mountain View,O=Google Inc,CN=mx.google.com": SMTP error from remote mail server after end of data: host ASPMX.L.GOOGLE.COM [74.125.136.27]: 550-5.7.1 [93.95.228.179      12] Our system has detected that this message is\n550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to Gmail,\n550-5.7.1 this message has been blocked. Please visit\n550-5.7.1 http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for\n550 5.7.1 more information. e2si17652766wij.118 - gsmtp

The /etc/mailname has been set to crin1.crin.org so the only reason I can think the HELO was crin1 is that exim4 hadn't been restarted, I have now done that so testing email again...

chris — Tue, 28 Apr 2015 12:28:36 GMT

Still getting:

2015-04-28 12:26:43 1Yn4b4-0002HK-HM ** chris@webarchitects.co.uk R=dnslookup T=remote_smtp X=TLS1.2:RSA_AES_128_CBC_SHA1:128 DN="C=GB,ST=South Yorkshire,L=Sheffield,O=webarchitects.coop,CN=mx.webarch.net": SMTP error from remote mail server after RCPT TO:<chris@webarchitects.co.uk>: host mx.webarch.net [81.95.52.71]: 550 Invalid HELO

Have edited /etc/hostname to crin1.crin.org and trying again...

chris — Tue, 28 Apr 2015 12:32:11 GMT

Still the same error:

2015-04-28 12:28:37 1Yn4cu-0002MR-6V ** chris@webarchitects.co.uk R=dnslookup T=remote_smtp X=TLS1.2:RSA_AES_128_CBC_SHA1:128 DN="C=GB,ST=South Yorkshire,L=Sheffield,O=webarchitects.coop,CN=mx.webarch.net": SMTP error from remote mail server after RCPT TO:<chris@webarchitects.co.uk>: host mx.webarch.net [81.95.52.71]: 550 Invalid HELO

Added:

primary_hostname='crin1.crin.org'

To /etc/exim4/update-exim4.conf.conf and ran update-exim4.conf and service exim4 restart.

See if that worked...

chris — Tue, 28 Apr 2015 12:41:48 GMT

Have uncommented these lines in /etc/exim4//exim4.conf.template and run update-exim4.conf and service exim4 restart and testing again, if this doesn't work I'll install postfix.

chris — Tue, 28 Apr 2015 12:44:15 GMT

Progress:

2015-04-28 12:41:49 1Yn4pg-0002tH-C0 ** chris@webarchitects.co.uk R=dnslookup T=remote_smtp: SMTP error from remote mail server after HELO 'crin1.crin.org': host mx.webarch.net [81.95.52.71]: 501 Syntactically invalid HELO argument(s)

Edited these two files, ./exim4.conf.localmacros ./update-exim4.conf.conf to remove the single quotes...

chris — Tue, 28 Apr 2015 12:46:37 GMT

Emailing of tickets and comments out now works!

I'll next install the Estimation and Time Tracking plugin, http://trac-hacks.org/wiki/TimingAndEstimationPlugin

chris — Tue, 28 Apr 2015 13:03:23 GMT

Installing http://trac-hacks.org/wiki/TimingAndEstimationPlugin

su - trac -s /bin/bash
svn co http://trac-hacks.org/svn/timingandestimationplugin/branches/trac1.0
cd trac1.0/
python setup.py bdist_egg
cp dist/timingandestimationplugin-1.4.6-py2.7.egg /var/www/trac/plugins/

Edit ~/conf/trac.ini and add to the end of the file:

[components]
timingandestimationplugin.* = enabled

hours changed; totalhours, billable set

chris — Tue, 28 Apr 2015 13:04:41 GMT

hours changed from 0 to 4
totalhours set to 4
billable unset

Recording time spent so far today to test the timing plugin.

hours, totalhours changed

chris — Tue, 28 Apr 2015 13:21:14 GMT

hours changed from 0 to 0.25
totalhours changed from 4 to 4.25

To get the plugin working I also had to run:

trac-admin /var/www/trac upgrade

Some other config changes in ~/conf/trac.ini:

[ticket]
default_owner = chris
[trac]
base_url = https://trac.crin.org/trac
secure_cookies = true
[timeline]
changeset_show_files = 1
default_daysback = 90
max_daysback = 3650
ticket_show_component = true
ticket_show_details = true

Still not seeing comments show on the Timeline, not sure why, see http://trac.edgewall.org/ticket/6519 but that can wait...

hours, totalhours changed

chris — Wed, 29 Apr 2015 13:01:37 GMT

hours changed from 0 to 0.25
totalhours changed from 4.25 to 4.5

Some tweaks made to the setup today.

Comments now show on the Timeline if "Ticket updates" is ticked, https://trac.crin.org/trac/timeline

The CRIN favicon was downloaded into /var/www/trac/apache/htdocs/site/ and referenced from trac.ini:

[project]
icon = site/favicon.ico

The /etc/apache2/envvars file had the following added to it:

# http://trac.edgewall.org/wiki/0.12/TracInstall#RunningtheStandaloneServer
export PKG_RESOURCES_CACHE_ZIP_MANIFESTS=1

The Apache VirtualHost was changed to get another site working:

<VirtualHost *:80>
<VirtualHost *:443>

hours, totalhours changed

chris — Wed, 29 Apr 2015 15:05:34 GMT

hours changed from 0 to 0.15
totalhours changed from 4.5 to 4.65

Enabling email2trac, which would allow tickets to be commented on via email will need a SMTP account to use, at the moment crin.org email is done by Google. So we either need to create an account there or I could create and use a (tmp?) account on the Webarchitects mailserver (I don't think one can be created at 1984.is as there isn't a hosting package on the account).

I don't want to enable incomming email on crin1 as we would then have the overhead or running anti-spam applications.

The best thing to do for now might be to set a MX record for trac.crin.org and set it up on the Webarchitects mailserver as I know how to do this whereas I haven't used Google for this.

hours, totalhours changed

chris — Wed, 29 Apr 2015 15:52:03 GMT

hours changed from 0 to 0.35
totalhours changed from 4.65 to 5.0

MX record created and also email account.

aptitude install fetchmail

Edit /etc/email2trac.conf:

[DEFAULT]
project: /var/www/trac

Create a ~/.fetchmailrc:

poll mail.webarch.net with proto IMAP and options no dns
port 993
auth password
user 'trac.trac.crin.org' there with password 'XXX' is 'trac' here options ssl
sslfingerprint '93:4C:E0:98:B4:89:84:4F:A4:ED:45:15:51:A5:AB:F2'
mda "/usr/bin/email2trac"

And it seems to run fine:

su - trac -s /bin/bash
fetchmail
fetchmail: No mail for trac.trac.crin.org at mail.webarch.net

So going to try repliying to this via email...

chris — Wed, 29 Apr 2015 15:54:08 GMT

This is a test reply by email.

hours, totalhours changed

chris — Wed, 29 Apr 2015 16:06:34 GMT

hours changed from 0 to 0.17
totalhours changed from 5.0 to 5.17

Each year when the mail.webarch.net TLS certificate is updated we will need to update the ~/.fetchmailrc files with the new one, you can get it like this:

fetchmail -v -p imap -u trac.trac.crin.org mail.webarch.net | grep -i fingerprint

The email worked but I noticed the times are wrong:

aptitude install rdate
date ; rdate ntp.demon.co.uk ; date
Wed Apr 29 15:57:14 GMT 2015
Wed Apr 29 15:58:34 GMT 2015

So ntp was installed:

aptitude install ntp

And it seems to be running OK:

service ntp status
* ntp.service - LSB: Start NTP daemon
   Loaded: loaded (/etc/init.d/ntp)
   Active: active (running) since Wed 2015-04-29 16:00:07 GMT; 24s ago
   CGroup: /system.slice/ntp.service
           `-24809 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 112:117
Apr 29 16:00:07 CRIN1 ntpd[24809]: proto: precision = 0.144 usec
Apr 29 16:00:07 CRIN1 ntpd[24809]: Listen and drop on 0 v4wildcard 0.0.0.0 UDP 123
Apr 29 16:00:07 CRIN1 ntp[24802]: Starting NTP server: ntpd.
Apr 29 16:00:07 CRIN1 ntpd[24809]: Listen and drop on 1 v6wildcard :: UDP 123
Apr 29 16:00:07 CRIN1 ntpd[24809]: Listen normally on 2 lo 127.0.0.1 UDP 123
Apr 29 16:00:07 CRIN1 ntpd[24809]: Listen normally on 3 eth0 93.95.228.179 UDP 123
Apr 29 16:00:07 CRIN1 ntpd[24809]: Listen normally on 4 lo ::1 UDP 123
Apr 29 16:00:07 CRIN1 ntpd[24809]: Listen normally on 5 eth0 fe80::5054:5dff:fe5f:e4b3 UDP 123
Apr 29 16:00:07 CRIN1 ntpd[24809]: peers refreshed
Apr 29 16:00:07 CRIN1 ntpd[24809]: Listening on routing socket on fd #22 for interface updates

Also did this for crin2.

Some email settings were changed in conf/trac.ini:

[notification]
always_notify_owner = true
always_notify_reporter = true

And I'm going to test with another email...

chris — Wed, 29 Apr 2015 16:08:41 GMT

This is another email test.

hours, totalhours changed

chris — Wed, 29 Apr 2015 16:15:22 GMT

hours changed from 0 to 0.1
totalhours changed from 5.17 to 5.27

Set up a crontab for the trac user:

# m h  dom mon dow   command
*/5 * * * * fetchmail

Although email in is now working email out has stopped, I have changed these settings in trac.ini to try to solve this:

[notification]
smtp_default_domain = crin1.crin.org
use_public_cc = true

hours, totalhours changed

chris — Wed, 29 Apr 2015 16:18:25 GMT

hours changed from 0 to 0.05
totalhours changed from 5.27 to 5.32

I have also changed:

sendmail_path = /usr/sbin/sendmail

Still not seeing any email being generated by tickets.

hours, totalhours changed

chris — Wed, 29 Apr 2015 16:27:09 GMT

hours changed from 0 to 0.08
totalhours changed from 5.32 to 5.4

Tried changing these again:

smtp_default_domain = trac.crin.org
use_tls = false

hours, totalhours changed

chris — Wed, 29 Apr 2015 16:32:59 GMT

hours changed from 0 to 0.1
totalhours changed from 5.4 to 5.5

That fixed it must have been use_tls = false, exim4 however did send the email using TLS in any case:

Received: from crin1.crin.org ([93.95.228.179])
        by elderberry.rat.burntout.org with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128)
        (Exim 4.80)
        (envelope-from <trac@trac.crin.org>)
        id 1YnUpK-0004B0-Gu
        for chris@webarchitects.co.uk; Wed, 29 Apr 2015 17:27:12 +0100
Received: from localhost ([::1] helo=crin1.crin.org)
        by crin1.crin.org with esmtp (Exim 4.84)
        (envelope-from <trac@trac.crin.org>)
        id 1YnUpJ-0006XB-HB; Wed, 29 Apr 2015 16:27:09 +0000

And fetchmail uses TLS via it's 5 mins crontab.

So transport layer encrypted email in and out of Trac is all working now.

chris — Wed, 29 Apr 2015 17:00:39 GMT

This is test of the crontab running fetchmail -- if this appears as a
comment it is working.

billable changed

chris — Wed, 29 Apr 2015 17:02:48 GMT

billable set

hours, totalhours changed

chris — Thu, 07 May 2015 10:12:19 GMT

hours changed from 0 to 0.25
totalhours changed from 5.5 to 5.75

Add account for Graham:

sudo -i
su - trac -s /bin/bash
htdigest .htpasswd trac graham

I'm not sure if Graham will need Trac admin permissions, if he does we can run:

trac-admin /var/www/trac permission add graham admin
trac-admin /var/www/trac permission add graham TRAC_ADMIN

I have added this to wiki:Trac

hours, status, totalhours changed; resolution set

chris — Thu, 07 May 2015 10:42:02 GMT

hours changed from 0 to 0.25
status changed from accepted to closed
resolution set to fixed
totalhours changed from 5.75 to 6.0

I have done some documentation of Trac at wiki:Trac and thinks this ticket is now good to close.

chris — Thu, 14 May 2015 18:45:59 GMT

After editing /etc/aliases on both servers and rebooting them, see ticket:6#comment:18 Trac stopped sending email, this is the error displayed:

Warning: The change has been saved, but an error occurred while sending notifications: SMTP server connection error ([Errno 101] Network is unreachable). Please modify [notification] smtp_server or [notification] smtp_port in your configuration.

So /var/www/trac/conf/trac.ini was edited and this:

email_sender = SmtpEmailSender

Was changed to:

email_sender = SendmailEmailSender

And if a email goes out about this comment the above changes have fixed outgoing Trac email.

hours, totalhours changed

chris — Thu, 14 May 2015 18:54:25 GMT

hours changed from 0 to 0.1
totalhours changed from 6.0 to 6.1

Outgoing Trac email isn't fixed. this is the error when submitting ticket:1#comment:27:

Warning: The change has been saved, but an error occurred while sending notifications: Sendmail failed with (1, 2015-05-14 18:45:59 unable to set gid=114 or uid=0 (euid=0): forcing real = effective), command: '[u'/usr/sbin/sendmail', '-i', '-f', u'trac@…', u'chris@…', u'jonas@…']'

In an attempt to fix this in /var/www/trac/conf/trac.ini smtp_server was changed from localhost into:

smtp_server =

Given the error above I'm not sure this will fix the problem...

chris — Thu, 14 May 2015 18:59:17 GMT

Same error, testing again...

chris — Thu, 14 May 2015 19:05:11 GMT

In /var/log/exim4/mainlog we have:

2015-05-14 19:01:26 socket bind() to port 25 for address ::1 failed: Cannot assign requested address: waiting 30s before trying again (9 more tries)

So I think this is a result of IPv6 being disabled following the reboot, so dpkg-reconfigure exim4-config to disable IPv6.

If I'm right regarding the problem this comment should result in a email.

chris — Thu, 14 May 2015 19:06:48 GMT

That didn't fix it, same error.

chris — Thu, 14 May 2015 19:09:16 GMT

Changing these variables back in /var/www/trac/conf/trac.ini:

email_sender = SmtpEmailSender
smtp_server = localhost

And testing to see if a email is sent...

hours, totalhours changed

chris — Thu, 14 May 2015 19:10:11 GMT

hours changed from 0 to 0.15
totalhours changed from 6.1 to 6.25

Last edit:

use_tls = false

This should do it.

chris — Thu, 14 May 2015 19:15:30 GMT

So the Trac sending email problem was just down to IPv6 being disabled and nothing to do with Exim or Trac settings -- the following had been added to /etc/sysctl.conf:

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

To disable IPv6 since we are not using it at the moment (the firewall is only set for IPv4 etc) and this only took effect after the server was rebooted due to Venom.

cc changed

chris — Thu, 18 Jun 2015 11:56:28 GMT

cc jenny gillian added; jonas removed

CCs changed.