Install and Configure Varnish for Nginx and Magento 2 on Ubuntu 16.04

Ref:https://blog.pusher.com/speeding-up-your-website-using-varnish/
https://www.cyberciti.biz/faq/how-to-install-and-configure-varnish-cache-on-ubuntu-linux-16-04-lts/
https://www.howtoforge.com/tutorial/ubuntu-nginx-varnish/
https://amasty.com/blog/use-varnish-with-magento-2-for-better-end-user-experience/

Install Varnish:

Start/stop/restart the varnish service:

Check the varnish status (when active):

Varnish installed and it is a default configured to listen on port 6081 and 6082 for varnish admin:

When we open on the browser: http://localhost:6081/, we’d get a varnish error pageHowever http://localhost:6082/ will show page error.
We will be changing this so it listens on port 80 (fontaineind use the default port 6081 for varnish)  and Nginx listens on port 8080. To do this, edit your site’s Nginx configuration file. In our case, this file is /etc/nginx/sites-available/fontaineind.test.

modify

Save the file. Then restart nginx server:

When you open your website with the port 8080, http://fontaineind.test:8080, It’d show the website but the css seems missing! I think I need to clear the cache and rebuild the cache. but it show an expected error.

Another error when trying to show the magento 2 command:

Solution: THIS IS BECAUSE I USE PHP 7.2. CHANGE IT TO PHP 7.0

Then the magento 2 command is running well back.

also probably I need to change the base url to include the port 8080? –> NO NEED. we will take it later. Let’s continue.

Now that port 80 is free (NO. I NEED TO CHECK IT VIA sudo netstat -plntu | grep ‘:80’), let’s configure Varnish to be the one that responds. Open the file /etc/default/varnish for editing. Here is the default content.

Look for the line below:

change ‘-a :6081’ to ‘-a :80’:

Save the file.
NOTE: I also can change the malloc. malloc is RAM usage for Varnish. Default is 256m. I can change it to (for example) 1024m (1G)

Next, open the file /etc/varnish/default.vcl and check that the default backend is set to port 8080, because this is where Nginx serves from now. Look for the line below and make sure the .port is 8080. Here is the default content:

NOTE: For the explanation of vcl_recv, vcl_backend_response and vcl_deliver, pls read https://amasty.com/blog/use-varnish-with-magento-2-for-better-end-user-experience/
make sure the backend port is 8080

Once you have confirmed this, the next thing will be to copy the varnish.service file to our systemd directory. This will enable systemd to start varnish on port 80. Run the command below:

This will copy the varnish service file, and then open an editor with the contents.

Look for the ExecStart line:

Now change the -F -a :6081 to -F -a :80. Leave the localhost:6082 on the same line as it is. Now save and exit the file. Run the commands below to reload everything you have just configured:

The commands above will reload the system daemon, reload varnish, restart the varnish service, and restart Nginx.
NOTE: I need to check the varnish status as well. Varnish can’t start because port 80 already is use:

Check who is using port 80:

it seems nginx with pid 19110 is using both port 80 and 8080. or here is the complete list:

Solution: nginx use both ports 80 and 8080 because in sites-enabled directory there are many nginx setting for different websites that it still use port 80.

SO. REPLACE THE PORT 80 TO 8080 FOR ALL WEBSITE IN SITES-ENABLES DIRECTORY!!!
AFTER I CHANGED THE PORT IN ALL THE WEBSITE, NO MORE APPLICATION USE PORT 80 ANYMORE

OR

Nginx is using port 8080 now. I can start varnish and check the status:

NOTE: TO RUN THE OTHERS RUNNING WEBSITE (WITHOUT VARNISH RUINNING) THAT IT HAD BEEN WORKING PREVIOUSLY, I NEED TO ADD THE PORT 8080 AT THE END OF THE URL. FOR EXAMPLE:
http://localhost:8080/phpinfo.php
http://notarisfemira.test:8080/
http://phpmyadmin.test:8080

Now visit the IP of your web page and you should see nothing has changed. However, when you run it through the HTTP checker we used before, you should see that the headers have changed.
http://fontaineind.test/ –> First time run about 20 seconds

The second run (refresh) about 9 seconds

NOTE: I can see some variables like X-Varnish and Age. But why Age still 0 on the refresh page? It means it miss the cache.
In magento, I already enabled Page Cache and Block HTML cache.

Configure Magento to use Varnish
To configure Magento to use Varnish:

Log in to the Magento Admin as an administrator.
Click STORES > Configuration > ADVANCED > System > Full Page Cache.
From the Caching Application list, click Varnish Cache (recommended).
Enter a value in the TTL for public content field.
Expand Varnish Configuration and enter the following information:
Access list: localhost   –> 172.31.31.224 for fontaineind.com.au
Backend host: localhost   –> 172.31.31.224 for fontaineind.com.au
Backend port: 8080
Then click a button ‘Export VCL for Varnish 4’ -> It’d create a file ‘varnish.vcl’ and download it.
Here is the file content:

SO I need to modify the content of /etc/varnish/default.vcl according to this varnish.vcl.

Just copy and paste all content of varnish.vcl to all content in /etc/varnish/default.vcl. Then restart varnish: sudo service varnish restart
NOTE: IN THE DEV SERVER, I PREFER TO DISPLAY THE X-VARNISH AND AGE VARIABLES. SO I MODIFIED /etc/varnish/default.vcl LIKE THIS:

I also modified /home/teddy/Documents/works/fontaineind/app/etc/env.php to include the varnish setting:

clear and flush the cache. BUT Age still 0 !!!! EVEN WEIRDS, WHEN I CHANGED IN THE MAGENTO ADMIN THAT CACHE APPLICATION IS VARNISH THEN REFRESH THE WEBSITE (ALSO RESTART VARNISH), I GOT THE PAGE LOAD SLOWER!!! WHY?? –> SOLUTION: I NEED TO EXPORT VCL FILE LIKE ABOVE AND COPY THE CONTENT INTO /etc/varnish/default.vcl

NOW THE VARNISH IS WORKING!!!
THE SECOND AND MORE REFRESH WOULD MAKE THE PAGE LOAD FASTER. HERE IS THE RESPONSE:

SEE THE AGE = 42 AND X-Varnish: 131131 3