Question: Q2. Build a Big Data Processing Pipeline using Databricks and Azure (20 Marks) You are a data analytics professional supporting a local photography competition. During

Q2. Build a Big Data Processing Pipeline using
Q2. Build a Big Data Processing Pipeline using Databricks and Azure (20 Marks) You are a data analytics professional supporting a local photography competition. During the competition photographs submitted will be uploaded to a cloud storage container in the cloud, you are tasked with making an automated data pipeline which will periodically scan all images uploaded using AI based cloud services and extract meta data about the photographs submitted to a database which can be viewed by the judges and the organisers. Build a cloud-based pipeline using components from Microso Azure and Databricks with a complementing Visual Dashboard in Power BI. a) Provide a narrative with screenshots to highlight your proposed solution the components involved and how it works. Also note any other considerations that you have in terms of the pros and cons of your approach. (15 Marks) b) Discuss 5 insights that you can derive from the dashboard, use with appropriate visuals to reinforce your insights

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related General Management Questions!