This is obviously relevant for web applications and websites in general, which are available to the public. In such situations, the typical solution is to purchase an SSL certificate from a certification authority (such as RapidSSL or GoDaddy) and enable it within the webserver. This article is not about this.

What we’re going to talk about today is the security of Intranet web applications, such as a websites which are not publicly available, but somehow available to specific people having access to private networks.

Why is it important to secure intranet applications too?

Somebody might think that, since an application is not publicly available, it is not important to secure it with SSL. This is not true, and here’s why.

Think about a document storage web application that is running in the local area network (LAN) of your company. That application of course may contain many documents of different nature, such as customers’ and suppliers’ invoices and employees contracts and payment receipts.
While some of this documents might be visible to all the users of such web application, many others have to be visible to specific users only.
Let’s say that you implement a very secure ACL to ensure this security layer on your application and avoid unauthorized access to your sensitive data.
If you don’t serve your project using the HTTPS protocol, than all your documents are traveling in the cables of your LAN unencrypted, and anybody connected to the same network can easily scan your network and read those documents. Plus, a malicious user could use the same technique to sniff user passwords and easily access your secure web app with other peoples’ credentials (don’t you believe that? Check this tutorial).

That said, I hope you are all convinced now that SSL is important for every web application, even if it’s not public.

How to secure an intranet application

There are a few options to secure private web applications.

1. The dumbest option is to generate a self signed certificate. That does not solve the security problem though (read this to understand why).
2. Buy a standard certificate. This is perfectly acceptable but requires spending some amount of money every year.
3. Generate a certificate using Letsencrypt

If you haven’t heard about Letsencrypt before, go have a look at their website to learn more. In one sentence, it is an organization that aims at making the web more secure and gives you free SSL certificates for personal or professional use.
This is in my opinion the perfect solution for intranet applications (which tipically have budget constraints).

Generating SSL certificates with letsencrypt

Letsencrypt uses the ACME protocol to release SSL certificates in an automated manner.

That means you need an ACME client to get an SSL certificate. The official client is called certbot, it’s free and open source and it is easily installable in many UNIX-based operating systems. We’re not going to cover the installation of certbot, though, because it is out of scope. You can find easy tutorials for your operating system on the officiale page.

There are a few different ways to get a new certificate using certbot (such as apache, nginx, webroot, and so on. Read them here) but all of them have a very important common step to be performed which is domain ownership verification. This step is required by the certification authority to ensure that you are the owner, or at least you are allowed to manage, the DNS records for the internet domain you are requiring the certificate for. Skipping this step would mean that anybody could request a certificate for google.com or paypal.com and that of course would be a giant security problem.

How does Letsencrypt check domain ownership?

There are basically 2 ways:

• HTTP challenge: an HTTP request is sent to the domain name you requested the domain for. The server must respond to a specific endpoint with a specific content in order for the domain ownership to be verified.
• DNS challenge: letsencrypt servers will search on the DNS records of the requested domain for a specific TXT record, with a specific value.

HTTP challenge is the most common and the easiest, because certbot (and other ACME clients) can do that for you automatically. Unluckily, it requires that the domain name resolves to a public IP address which is freely reachable on the internet. But, remember, our domain is in the intranet and so this approach simply would not work in our scenario. That means we need to choose the DNS challenge option.

Generate a certificate using the DNS challenge

To do that, we need to use the manual strategy of certbot, by running a command similar to the following:

certbot -d private.mydomain.com --manual --preferred-challenges dns certonly

Of course adjust private.mydomain.com with your domain.
This command starts an interactive wizard:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges: dns-01 challenge for private.mydomain.com
------------------------------------------------------------------
NOTE: The IP of this machine will be publicly logged as having requested this certificate. If you're running certbot in manual mode on a machine that is not your server, please ensure you're okay with that.
Are you OK with your IP being logged?
------------------------------------------------------------------
(Y)es/(N)o: Y

------------------------------------------------------------------
Please deploy a DNS TXT record under the name _acme-challenge.private.mydomain.com with the following value:

LngHg_3lhXDJ_m3ArGTgtalz50uVCjXW5-zFVCulK8I

Once this is deployed,
------------------------------------------------------------------
Press Enter to Continue

At this point, as clearly requested by the script, you need to create a new TXT record in your DNS administration panel for _acme-challenge.private.mydomain.com with value LngHg_3lhXDJ_m3ArGTgtalz50uVCjXW5-zFVCulK8I

Once done, check that the DNS record propagated successfully (you can use this website to do that) than press enter to continue executing the script.

If all went well, you should see a success message:

IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at /etc/letsencrypt/live/private.mydomain.com/fullchain.pem.
Your cert will expire on 2017-05-16. To obtain a new or tweaked version of this certificate in the future, simply run certbot again.
To non-interactively renew *all* of your certificates, run "certbot renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le

Good, our certificates are ready! You can find everything you need under the folder: /etc/letsencrypt/live/mydomain.com/ (of course adjust the path to meet your domain name). In that folder you’ll find 4 files:

• cert.pem: the actual certificate
• chain.pem: the certificate of the certification authority
• fullchain.pem: basically the merge of the above 2 files
• privkey.pem: your private key (never share this file with anybody)

Now you only have to tell your webserver to use the HTTPS protocol and where to get the certificate files. Check the official documentation for Nginx and apache to get started.

Recap

In this article we discussed about the risks of having a plain HTTP web application hosted inside our company’s private network and the need of an SSL certificate to make traffic flow securely inside our ethernet wires.

We then learned how to generate a free SSL certificate using Letsencrypt even if our webservers are not public.

The problem

Let’s say you are also developing a PHP application and you want to use docker in your development machine. You will notice that – if you use the official PHP images from the docker hub – sending e-mail out of the box simply won’t work. Lets’ see what I mean in practice, considering this simple index.php file, sending out an e-mail:

$result = mail("info@example.com", "Subject", "The very important email body");

if ($result) {
    echo "Email sent correctly";
} else {
    echo "Error sending the email";
}

Please note: using the mail command to send an email is the simplest way, but there are many more powerful libraries out there that could make things easier. We have chosen to use the mail command just to keep the example as simple as possible.

If we run it using docker with the following command:

docker run -p "80:80" -v `pwd`:/var/www/html --rm php:7-apache

and visit http://localhost with our browser, we’ll see that the mail command will fail, and we’ll get an “Error sending the email” message.

The reason why this happens is that the “mail” command relies on the “sendmail” system command inside the container, which is missing from the official PHP docker image.

While you could simply create a new image extending the original one and adding the sendmail command, I suggest using a different strategy.

Here at MV Labs we do use a tool called Mailhog (https://github.com/mailhog/MailHog) which acts as an SMTP server listening on port 1025 but instead of sending real emails to your recipients, it simply displays them in a handy web interface (which is listening, by default, on port 8025). This way you avoid the risk of sending your testing e-mails to the customer (it happened at least once to everybody) and you don’t have to deal with spam filters or other annoyances in the e-mail chain.

So, how can we achieve this? To make things easier, we can take advantage of docker-compose. Let’s start by adding the mailhog container to our services:

version: '2'
services:
  php:
    image: php:7-apache
    volumes:
      - .:/var/www/html/
    ports:
      - 80:80
    networks:
      - base
  mailhog:
    image: mailhog/mailhog
    ports:
      - 8025:8025
    networks:
      - base
networks:
  base:

Now, if we run docker-compose up and head our browsers to http://localhost:8025 we should see the mailhog simple but powerful web interface.

We’re not done yet, though, as we still need to tell the mail command to send emails through our new mailhog smtp server. Unluckily this requires the building of a custom docker image, since PHP doesn’t allow using the native mail command to connect to a SMTP server. For this reason, we need a tool called ssmtp (https://linux.die.net/man/8/ssmtp) which is basically a configurable sendmail command replacement, and which forwards the message to a SMTP server configured in its config file. We’ll need the following Dockerfile:

FROM php:7-apache

# install ssmtp
RUN apt-get update; apt-get install ssmtp -y

# tell php to use ssmtp's sendmail for email sending
RUN echo "sendmail_path=/usr/sbin/sendmail -t -i" >/usr/local/etc/php/conf.d/sendmail.ini

# tell ssmtp to use mailhog as the mail transport
RUN echo "Mailhub=mailhog:1025" > /etc/ssmtp/ssmtp.conf
RUN echo "FromLineOverride=Yes" >> /etc/ssmtp/ssmtp.conf

Then we’ll need to tell docker-compose.yml to build the image from the Dockerfile for our PHP container:

version: '2'
services:
  php:
    build: .
    volumes:
      - .:/var/www/html/
    ports:
      - 80:80
    networks:
      - base
  mailhog:
    image: mailhog/mailhog
    ports:
      - 8025:8025
    networks:
      - base
networks:
  base:

and build our containers again with:

docker-compose up -d --build

Now we’re ready to send our first “virtual” email. Just head your browser to http://localhost and you’ll get the message “Email sent correctly”. If you check your mailhog at http://localhost:8025 you’ll see your email!

Summary

Using a simple example, we have demonstrated one possible solution for sending test e-mails out of our web applications under development in a secure and handy way, by using some nice open source tools such as SSmtp and Mailhog inside a docker dev environment.

Let us know what you think in the comments area below and happy developing!

The Problem

Script wasn’t outputting nor logging anything, so in order to find out more about the problem, I thought about logging last script execution datetime myself. I wanted to find out whether the problem was related to cron execution, or rather to some issue within the script itself. So I added following two lines (on top to the script):

$s_content = 'Script last run: '. date("d/m/Y h:i:s" . "\n");
file_put_contents('/tmp/cert_import.log', $s_content);

And executed it:

php ./importa_certificati.php

All I got was a nasty error message:

File size limit exceeded

At first I thought the problem might’ve been related to the specific file name I had picked (IE /tmp/cert_import.log). Did it exist already and was it already too big? I was sure I had checked things out. And actually, file wasn’t there upon a second check. So what was PHP ranting about? Could have it been a permission issue? Nope, script was being executed by root and was executable. So, where did the problem lie? I started to search for the problem by reverting the script to its original form, commenting out my additions:

// $s_content = 'Script last run: '. date("d/m/Y h:i:s" . "\n");
// file_put_contents('/tmp/cert_import.log', $s_content);

Problem magically disappeared. Weird, I thought. “Let’s see what happens if I prepare my string, but don’t it write to the file, I said to myself. I’m sure the problem lies there. Let’s check”:

$s_content = 'Script last run: '. date("d/m/Y h:i:s" . "\n");
// file_put_contents('/tmp/cert_import.log', $s_content);

File size limit exceeded

Whaat?!? Problem still there, despite no attempt to write to a file is even made. Things start to look awkward.

The Cause

What I had just discovered was that problem didn’t have anything to do with the file I was trying to write, but was related instead to the PHP process itself. In fact, wondering what other file size limit might’ve been reached, I came to think about logging. So, I opened php.ini and checked things out. Logging was configured indeed, pointing at /usr/local/zend/var/log/php.log through following directives:

; http://php.net/log-errors
log_errors = On

; Set maximum length of log_errors. [...]
log_errors_max_len = 1024

; Log errors to specified file. PHPs default behavior is to leave this value
; empty.
error_log = "/usr/local/zend/var/log/php.log"

I changed my working directory to /usr/local/zend/var/log/ and checked things out with ls:

[root@XXX ~]# cd /usr/local/zend/var/log/
[root@XXX log]# ls -alh
-rw-rw---- 1 apache zend 2.0G Dec 27 17:30 php.log

That exact 2Gb size looked suspicious, to say the least. Especially because the php.ini log_errors_max_len directive above had a 1024 value, not a 2048 one. So, I moved the log file out of its way, and gzipped it (so to save some space, without losing anything).

mv php.log
gzip php.log

I then run the script again. All went well this time, with no error being reported. Problem was caused by log file size indeed. Most likely no one cared about rotating it, believing that log_errors_max_len would do the job. As it’s explained on stackoverflow though, despite its possibly ambiguous name, log_errors_max_len only deals with the size of a single log message, not with the size of the log file itself.

Newly introduced problem was solved. But I was still wondering what might’ve caused the problem from the lines I had added in first place. Luckily I now had logs to check.

The timezone not set and include underlying issues

I opened the new php.log file and found a few entries like the following:

[05-Dec-2012 23:00:04 UTC] PHP Warning:  strtotime(): It is not safe to rely on the system's timezone settings. 
You are *required* to use the date.timezone setting or the date_default_timezone_set() function. 
In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. 
We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /[...]/importa_certificati.php on line 14

Easy problem to fix. And also found the reason why log file filled up so quickly. Such message was written for every run of another script where the date function was used.

Still, I hadn’t solved my problem yet. Script would run from console, but wouldn’t from cron. Again, logs proved to be my friends, with entries like the following:

[27-Dec-2012 23:09:02 UTC] PHP Warning:  include_once(../classi/dbconnect.php): failed to open stream: No such file or directory in /[...]/importa_certificati.php on line 3
[27-Dec-2012 23:09:02 UTC] PHP Warning:  include_once(): Failed opening '../classi/dbconnect.php' for inclusion (include_path='.:/[...]') in /[...]/importa_certificati.php on line 3
[27-Dec-2012 23:09:02 UTC] PHP Warning:  include_once(../classi/utils.php): failed to open stream: No such file or directory in /[...]/importa_certificati.php on line 4
[27-Dec-2012 23:09:02 UTC] PHP Warning:  include_once(): Failed opening '../classi/utils.php' for inclusion (include_path='.:[...]') in /[...]/importa_certificati.php on line 4
[27-Dec-2012 23:09:02 UTC] PHP Fatal error:  Class 'utils' not found in /[...]/importa_certificati.php on line 10

I had just found out that problem was due to the script having relative includes in the following form:

<!--?php

include_once('../classes/dbconnect.php');
include_once('../classes/utils.php');

$s_pathCerts = __DIR__ . '/../rsync/data';

</pre-->

So everything worked when the script was tested by customer from local directory:

php ./importa_certificati.php

But once it was invoked by cron, it stopped working, since path wasn’t quite the same.
And, because no error messages related to the issue were collected in php.log, no one ever discovered about the problem with the script, and blamed cron settings instead.
Changing lines above tothe following solved the problem:

<!--?php

include_once(__DIR__ . '/../classes/dbconnect.php');
include_once(__DIR__ . '/../classes/utils.php');

$s_pathCerts = __DIR__ . '/../rsync/data';

</pre-->

Things to take home from this experience

1. If php can not write to its log file it will stop script execution, without giving a clear explanation of what’s going on
2. The log_errors_max_len directive is a bit ambiguous. It deals with each log record, not with the whole file size
3. You always need to set your timezone, if you don’t want to have your logs filled with crap
4. When including stuff, it’s wise to rely on the php __DIR__ or dirname(__FILE__) constructs rather than on relative paths. You can’t be sure the scritpt will be invoked from where you expect.
5. When troubleshooting it’s important to think “outside” the box. In this case the problem was related to the script itself, as much as it was related to its environment.
6. If you do write to logs, you need to check’em out every once in a while. Having their content e-mailed to you periodically is also a good idea

Elephant picture courtesy of http://www.flickr.com/photos/laughingsquid/2218075860/