Hello everyone,
My family started a holiday tradition two years ago, that every year on Thanksgiving we share things we are grateful for. Here is mine.
The DET community was created in May and since then I have met many people who are also passionate about data engineering. Today, we have over 5K members sharing knowledge and learning from each other. Thank you so much for your support! 🧡
🌟 Community Highlights
When chatting with William Guo back in August, I learned that Apache Airflow, while popular in the US, isn’t as widely adopted in China because it doesn't scale well for the workload of large Chinese enterprises, such as SHEIN. Netflix had a similar story. Netflix’s data platform team also once evaluated Airflow but decided to build its own data orchestration tool: Maestro.
At the upcoming DET Webinar on Friday, Nov 17th, Jun He (Staff Software Engineer at Netflix) will tell you everything you need to know about Maestro:
Why Netflix chose to build its own scheduler vs. leverage existing frameworks
Common data orchestration needs in data teams
How Maestro is able to support Netflix's scale
Powerful features in Maestro that make DE's life easier
👉🏼 Sign up for the webinar HERE.
The DEML Summit 2023 in October was a big success! 🎉
We had so many great presentations from hands-on ICs and practitioners. If you missed any talks, don’t worry, the recordings are available:
👉🏼 Check out the full agenda and watch recordings HERE.
(Thank you to Decube, Segment, Onehouse, and Keboola for sponsoring the conference!)
Have a great holiday and see you next time! ❄️
Cheers,
Xinran Waibel (LinkedIn)