Why Data Engineering?
After building on the web for nearly two decades, I'm focusing on work with data, python, and more interesting problems to solve.
by Yin Yin Chan 🤓 June 2026
TL;DR
I see this transition as a natural extension and deepening of what I do when building web apps: call an API, take the API data, and transform it for the app's needs.
As a data engineer, I would be going more in-depth and working with data at a much larger scale. I'm excited to move to the "back of house", be able to solve highly complex data problems, and to segue myself into meachine learning.
A segue to machine learning
Without giving you the runaround, the end goal is to be a machine learning engineer. Data engineering is a bridge I see that can get me from where I am not to my destination.
From what I understand, machine learning relies on the ingestion and processing of data. If that is the case, then what better place for me to start than at the root of it. When I learn something new, I like to understand it from the ground up. The gap between what I know from building web apps and what I need to know as a data engineer is close enough to take this one small leap at a time.
Here is a detailed diagram of my master plan:
I’m diving in
I’ve started a data engineering project that takes multiple datasets from LADOT’s open data, building an end-to-end data pipeline, and using a “big data” technology stack. Here is a link to the project repository if you’d like to follow along on my progress. I’ll also be posting on my website as I go.
More to come
While I’ve answered the question on the macro-level, I’ll be updating this page as I learn more about the subject.