Authors :
Weiyi Ma; Dongmei Zhang
Volume/Issue :
Volume 6 - 2021, Issue 11 - November
Google Scholar :
http://bitly.ws/gu88
Scribd :
https://bit.ly/2ZSJj2C
Abstract :
The use of the website will generate a large
number of access logs. With the development of the access
log collection and processing technology, the access logbased processing can obtain some useful decision data,
which can assist the operator in strategic decision-making
and monitoring. With the increase of companies and
network users, more and more website systems, the
amount of data generated is getting larger and larger, and
the processing of data is becoming more and more
difficult. Based on this problem, the system was developed.
The traffic information of the system comes from the user
log data collected by the Nginx server and the front-end JS
burying point, and then the collected log data is spliced
into the URL as parameters. This system mainly uses
Hadoop, Mapreduce, HDFS and other technologies, and
uses Idea as a development tool. After testing, the system
function is simple and convenient, in line with the
company's processing data requirements, can produce
higher profit value.
Keywords :
E-commerce log analysis; Hadoop; Flume; HDFS; MapReduce
The use of the website will generate a large
number of access logs. With the development of the access
log collection and processing technology, the access logbased processing can obtain some useful decision data,
which can assist the operator in strategic decision-making
and monitoring. With the increase of companies and
network users, more and more website systems, the
amount of data generated is getting larger and larger, and
the processing of data is becoming more and more
difficult. Based on this problem, the system was developed.
The traffic information of the system comes from the user
log data collected by the Nginx server and the front-end JS
burying point, and then the collected log data is spliced
into the URL as parameters. This system mainly uses
Hadoop, Mapreduce, HDFS and other technologies, and
uses Idea as a development tool. After testing, the system
function is simple and convenient, in line with the
company's processing data requirements, can produce
higher profit value.
Keywords :
E-commerce log analysis; Hadoop; Flume; HDFS; MapReduce
