Introduction

We will discuss here how to build a simple online visitor tracking system using Python Flask. Every website owner would like to know the progress to the count of the online visitors because it gives them overall idea on the website’s analytical measurements. It will also give them how to make improvements on the websites.

We will use here @app.before_request decorator which Flask API provides to track your visitor when a visitor lands on your website.

This example will give you an idea how to implement such system to track visitors who visit your website on daily basis.

Prerequisites

Python 3.8.0, Flask 1.1.1, MySQL 8.0.17, PyMySQL 0.9.3

Related Posts:

• CodeIgniter Online Visitor Tracking System

Creating MySQL Table

We need to create MySQL table to store visitor information.

In the following table we store how many times a visitor has visited application or website. We store visitor’s IP address, which URLs visitor visits, from which page or URL visitor navigates, if there is any query parameters in the URL, on which browser visitor opens the website.

We have also another flag whether the visitor is unique or not. So it depends on how you are going to implement the functionality. In this basic example I have not worked on this flag.

CREATE TABLE `visits_log` (
  `log_id` int unsigned NOT NULL AUTO_INCREMENT,
  `no_of_visits` int unsigned NULL,
  `ip_address` varchar(20) NULL,
  `requested_url` tinytext NULL,
  `referer_page` tinytext NULL,
  `page_name` tinytext NULL,
  `query_string` tinytext NULL,
  `user_agent` tinytext NULL,
  `is_unique` tinyint NOT NULL DEFAULT '0',
  `access_date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`log_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

Creating Project Directory

Create a project root directory called python-flask-mysql-online-visitor-tracker as per your chosen location.

We may not mention the project’s root directory name in the subsequent sections but we will assume that we are creating files with respect to the project’s root directory.

Configure Flask

We here configure application through flask framework. Create a file called app.py with the below code.

Here we need to assign secret key otherwise session will not work in Python. The secret key, ideally, should be in encrypted format.

We have also configured the session timeout – 30 minutes because flask expires session once you close the browser unless you have a permanent session.

Basically a session will exist for 30 minutes and in this 30 minutes a user’s visits will be unique for a particular URL. So to set the flag is_unique in table vistis_log you need to work on this.

from flask import Flask
from datetime import timedelta
	
app = Flask(__name__)
app.secret_key = "secret key"
app.config['PERMANENT_SESSION_LIFETIME'] =  timedelta(minutes=30)

Database Configuration

We create the below db.py Python script to setup the MySQL database configurations for connecting to database and storing visitor information into visits_log table.

We need to configure database connection with flask module and that’s why we have imported app module and setup the MySQL configuration with flask module.

Make sure to change the database configuration values according to your database setup.

from app import app
from flaskext.mysql import MySQL

mysql = MySQL()
 
# MySQL configurations
app.config['MYSQL_DATABASE_USER'] = 'root'
app.config['MYSQL_DATABASE_PASSWORD'] = 'root'
app.config['MYSQL_DATABASE_DB'] = 'roytuts'
app.config['MYSQL_DATABASE_HOST'] = 'localhost'
mysql.init_app(app)

Configure Utility

I will create one general purpose utility Python script that will provide some utility functions to be used throughout the whole application wherever required.

We create a file called config.php with the following source code.

We check the key DNT in the requested http header and if its value is 1 then we don’t track visitors otherwise we will track the visitors if other conditions also meet.

We ignore IPs, for example, localhost, 127.0.0.1 etc because from localhost you won’t get real visitors.

I have created another function that will help us whether we want to consider a user is unique for a particular time period. For example, as I had said earlier that we want to consider a visitor unique if he/she is active for 30 minutes on the same browser.

from flask import request, session

DNT_TRACK = True #False
IGNORE_IPS = set(['127.0.0.1'])

def is_tracking_allowed():
	#print(request.headers)
	if 'DNT' in request.headers and request.headers['DNT'] == 1:
		return False
	if request.remote_addr in IGNORE_IPS:
		return False
	return True

def track_session():
	if 'track_session' in session and session['track_session'] == True:
		return True
	else:
		return False

Log Visitors

Now we will log visitors into MySQL database table. We create a file called visitor.py.

We first check whether session tracking is allowed, i.e., if you are going to consider a visitor unique for a time period.

If session tracking is allowed then we check if the visitor is visiting the same page within 30 minutes. If visitor visits the same page then we don’t store the same information into database otherwise we store the details into database.

We have used session.modified = True otherwise you won’t be able to update the session key’s value.

If session tracking is not allowed then every time a visitor visits a page we store details into MySQL table. In this case your database table will grow unnecessarily.

import config
import pymysql
from db import mysql
from flask import request, session

def track_visitor():
	if not config.is_tracking_allowed():
		return
	else:		
		ip_address = request.remote_addr
		requested_url = request.url
		referer_page = request.referrer
		page_name = request.path
		query_string = request.query_string
		user_agent = request.user_agent.string
				
		if config.track_session():
			log_id = session['log_id'] if 'log_id' in session else 0
			no_of_visits = session['no_of_visits']
			current_page = request.url
			previous_page = session['current_page'] if 'current_page' in session else ''
			
			if previous_page != current_page:
				
				log_visitor(ip_address, requested_url, referer_page, page_name, query_string, user_agent, no_of_visits)
		else:			
			conn = None
			cursor = None
			
			session.modified = True
			
			try:				
				conn = mysql.connect()
				cursor = conn.cursor()
				
				log_id = log_visitor(ip_address, requested_url, referer_page, page_name, query_string, user_agent)
				
				#print('log_id', log_id)
				
				if log_id > 0:				
					sql = 'select max(no_of_visits) as next from visits_log limit 1'
					
					conn = mysql.connect()
					cursor = conn.cursor(pymysql.cursors.DictCursor)
					
					cursor.execute(sql)
					row = cursor.fetchone()
					
					count = 0
					if row['next']:
						count += 1
					else:
						count = 1
					
					sql = 'UPDATE visits_log set no_of_visits = %s WHERE log_id = %s'
					data = (count, log_id,)
					
					cursor.execute(sql, data)
					
					conn.commit()
					
					session['track_session'] = True
					session['no_of_visits'] = count
					session['current_page'] = requested_url				
				else:
					session['track_session'] = False
			except Exception as e:
				print(e)
				session['track_session'] = False
			finally:
				cursor.close()
				conn.close()
				
def log_visitor(ip_address, requested_url, referer_page, page_name, query_string, user_agent, no_of_visits=None):
	sql = None
	data = None
	conn = None
	cursor = None
	log_id = 0
	
	if no_of_visits == None:
		sql = "INSERT INTO visits_log(no_of_visits, ip_address, requested_url, referer_page, page_name, query_string, user_agent) VALUES(%s, %s, %s, %s, %s, %s, %s)"
		data = (no_of_visits, ip_address, requested_url, referer_page, page_name, query_string, user_agent,)
	else:
		sql = "INSERT INTO visits_log(ip_address, requested_url, referer_page, page_name, query_string, user_agent) VALUES(%s, %s, %s, %s, %s, %s)"
		data = (ip_address, requested_url, referer_page, page_name, query_string, user_agent,)
	
	try:				
		conn = mysql.connect()
		cursor = conn.cursor()
		
		cursor.execute(sql, data)
		
		conn.commit()
		
		log_id = cursor.lastrowid
		
		return log_id
	except Exception as e:
		print(e)
	finally:
		cursor.close()
		conn.close()

Track Visitors

We have already created the required functionality to store information into database table. Now we will see how to track visitors when he/she visits your web pages.

Create a Python script main.py with the following source code.

import visitor
from app import app
from flask import jsonify	
	
@app.before_request
def do_something_when_a_request_comes_in():
	visitor.track_visitor()

@app.route('/')
def home():
	return jsonify({'msg' : 'hello'})

		
if __name__ == "__main__":
    app.run()

In the above Python script we have used a decorator from Flask API @app.before_request which will trigger your function when a request comes in your web page. So you don’t need to write a hook but Flask API already provides for you to use on application.

We have another function that returns simple hello message when you hit the URL http://localhost:5000.

This is a simple application that will track your visitors. To track referrer page you need to have more navigation on your website.

Testing the Application

Now navigate to the project directory and execute the command python main.py, your server will be started on default port 5000.

If you want to change the port then you can change the line app.run() to app.run(port=5001), where 5001 is the new port.

Now you can hit the URL http://localhost:5000 on your browser and you will see visitors details are logged into MySQL table.

You can see I have accessed this application few times from different browsers such as Chrome, FireFox, Internet Explorer in the below output from visits_log table.

As I have said earlier this application has only one URL and that’s why you won’t see value for referrer page for most of entries. You should have few navigations on your website and users should navigate from one page to another page then only you get proper value for referrer page column.

Now you can create a UI or front-end where you can track your visitors’ statistics on a graph.

Source Code

Download

Thanks for reading.