Sticky post

Classify Images of Clouds in the Cloud with AutoML Vision

AutoML Vision helps developers with limited ML expertise train high quality image recognition models. Once you upload images to the AutoML UI, you can train a model that will be immediately available on GCP for generating predictions via an easy to use REST API. In this lab you will upload images to Cloud Storage and use them to train a custom model to recognize different types of clouds (cumulus, cumulonimbus, etc.). Set up AutoML Vision AutoML Vision provides an interface for all the steps in training an image classification model and generating predictions on it. Start by enabling the Cloud … Continue reading Classify Images of Clouds in the Cloud with AutoML Vision

Sticky post

Machine Learning with Spark on Google Cloud Dataproc

In this post you will learn how to implement logistic regression using a machine learning library for Apache Spark running on a Google Cloud Dataproc cluster to develop a model for data from a multivariable dataset. Google Cloud Dataproc is a fast, easy-to-use, fully-managed cloud service for running Apache Spark and Apache Hadoop clusters in a simple, cost-efficient way. Cloud Dataproc easily integrates with other Google Cloud Platform (GCP) services, giving you a powerful and complete platform for data processing, analytics and machine learning . Apache Spark is an analytics engine for large scale data processing. Logistic regression is available as a … Continue reading Machine Learning with Spark on Google Cloud Dataproc

Sticky post

Azure Platform for Data Engineers(part-2)

Explore data types: Azure provides many data platform technologies to meet the needs of common data varieties. It’s worth reminding ourselves of the two broad types of data: structured data and nonstructured data. Structured data In relational database systems like Microsoft SQL Server, Azure SQL Database, and Azure SQL Data Warehouse, data structure is defined at design time. Data structure is designed in the form of tables. This means it’s designed before any information is loaded into the system. The data structure includes the relational model, table structure, column width, and data types. Relational systems react slowly to changes in … Continue reading Azure Platform for Data Engineers(part-2)

Azure Platform for Data Engineers(part-1)

Over the last 30 years, we’ve seen an exponential increase in the number of devices and software that generate data to meet current business and user needs. Businesses store, interpret, manage, transform, process, aggregate, and report this data to interested parties. These parties include internal management, investors, business partners, regulators, and consumers. Data consumers view data on PCs, tablets, and mobile devices that are either connected or disconnected. Consumers both generate and use data. They do this in the workplace and during leisure time with social media applications. Business stakeholders use data to make business decisions. Consumers use data to … Continue reading Azure Platform for Data Engineers(part-1)

Sticky post

Cloud Engineering: Creating a Virtual Machine

Introduction: Google Compute Engine lets you create virtual machines running different operating systems, including multiple flavors of Linux (Debian, Ubuntu, Suse, Red Hat, CoreOS) and Windows Server, on Google infrastructure. You can run thousands of virtual CPUs on a system that has been designed to be fast and to offer strong consistency of performance. Here, you’ll learn how to create virtual machine instances of various machine types using the Google Cloud Platform (GCP) Console and using the gcloud command line. You’ll also learn how to connect an NGINX web server to your virtual machine. You should type the commands to reinforce their … Continue reading Cloud Engineering: Creating a Virtual Machine

Sticky post

How to Use AutoML and Vision API in GCP

What is the Vision API and what can it do? The vision API is an API that uses machine learning and other Google services to extract information from images. The sorts of predictions that it can currently make include but are not limited to the following list: Label Detection, which is used to detect the presence of certain broad classes of objects within images Text Detection, which can be used to extract text from images, a process that also is referred to as OCR Safe Search Detection, which can be used to check if an image is safe to serve … Continue reading How to Use AutoML and Vision API in GCP

BigQuery ML(move your model towards data and not data towards model)

Overview BigQuery ML enables users to create and execute machine learning models in BigQuery using standard SQL queries. BigQuery ML democratizes machine learning by enabling SQL practitioners to build models using existing SQL tools and skills. BigQuery ML increases development speed by eliminating the need to move data. BigQuery ML functionality is available by using: The BigQuery web UI The bq command-line tool The BigQuery REST API An external tool such as a Jupyter notebook or business intelligence platform Data Analyst is a Machine learning engineer now? Machine learning on large data sets requires extensive programming and knowledge of ML frameworks. These … Continue reading BigQuery ML(move your model towards data and not data towards model)

How to use Cloud Storage and Cloud SQL

In this post, you create a Cloud Storage bucket and place an image in it. You’ll also configure an application running in Compute Engine to use a database managed by Cloud SQL. For this lab, you will configure a web server with PHP, a web development environment that is the basis for popular blogging software. Outside this lab, you will use analogous techniques to configure these packages. You also configure the web server to reference the image in the Cloud Storage bucket. Objectives In this lab, you learn how to perform the following tasks: Create a Cloud Storage bucket and … Continue reading How to use Cloud Storage and Cloud SQL

Sticky post

Use Of Cloud IoT Core

Use IoT Core to create a registry Use IoT Core to create a device Use Stackdriver Logging to view device logs Enable APIs In this section, you check that all the APIs you will use in this lab are enabled. In the GCP Console, on the Navigation menu (), click APIs & Services. Scroll down and confirm that your APIs are enabled. Cloud IoT API Cloud Pub/Sub API Container Registry API If an API is disabled, click Enable APIs and services at the top, search for the API by name, and enable it for your project. Make sure you are in the correct Qwiklabs project. … Continue reading Use Of Cloud IoT Core

Sticky post

Using Pubsub to publish messages

Google Cloud Pub/Sub is a messaging service for exchanging event data among applications and services. A producer of data publishes messages to a Cloud Pub/Sub topic. A consumer creates a subscription to that topic. Subscribers either pull messages from a subscription or are configured as webhooks for push subscriptions. Every subscriber must acknowledge each message within a configurable window of time. the GCP console opens in this tab.Note: You can view the menu with a list of GCP Products and Services by clicking the Navigation menu at the top-left, next to “Google Cloud Platform”.  The Google Cloud Shell Activate Google Cloud Shell Google … Continue reading Using Pubsub to publish messages