Sticky post

insights from E-Commerce retail data set

We are using Bigquery as our data warehouse solution and using standard SQL as query language . For dataset we use Google’s Google Analytics logs of an merchants website. You need to enable your bigquery account which has a daily limit and there after it is cost effective. Click Navigation menu > BigQuery. Click Done. BigQuery public datasets are not displayed by default in the BigQuery web UI. To open the public datasets project, open https://console.cloud.google.com/bigquery?p=data-to-insights&page=ecommerce in a new browser window. In the left pane, in the Resource section, click data-to-insights. In the right pane, click Pin Project. Explore ecommerce data Problem :  Your data analyst team exported … Continue reading insights from E-Commerce retail data set

Sticky post

Analyzing logs in real time using Fluentd and BigQuery

This tutorial shows how to log browser traffic and analyze it in real time. This is useful when you have a significant amount of logging from various sources and you want to debug issues or generate up-to-date statistics from the logs. The tutorial describes how to send log information generated by an NGINX web server to BigQuery using Fluentd, and then use BigQuery to analyze the log information. It assumes that you have basic familiarity with Google Cloud Platform (GCP), Linux command lines, application log collection, and log analysis. Build dataprep skills real time with me here. Introduction Logs are a powerful … Continue reading Analyzing logs in real time using Fluentd and BigQuery