Web Log Analytics

  • Project Role: Lead Developer
  • Project date: 2013 - 2015
  • Project Specific Skills: Hive, HBase, Java, MapReduce, flume, D3.js, Spark, Spark streaming

Description

Log Analytics is an initiative by Client to be able to provide more statistical information about their website. The log files located on web servers are analysed to extract useful information. Java API is used to query the data (Number of hits, IP to location, most popular page, and most popular keyword found etc.) and display the results on web UI. Objective of the project is to be able to extract intelligence from more sources, faster than ever before by creating a scalable big data solution based on Hadoop, flume, HIVE, HBASE, Spark technologies as well as the capability to create a subset of data for reporting and analysis.