Data lakes can include at-rest and streaming data because these two data sources often need to be combined. By evaluating the tools within a big data ecosystem, the combination of these can be used.

For this assignment, you will utilize data from both concepts to gather incremental changes as they occur. While the data lake will not be populated, the architecture and design will consume and operate so that the destination could be many sources, including a data lake.

The project deliverables include the following:

Consume data from this link.

Consume the data locally on your PC using Python.

Construct a query algorithm that produces an incremental number of changes in real time. The Python code will require the use of a streaming library of your choosing.