Jun 23Embracing Diversity in Open Source: Apache Iceberg CommunityOpen-source communities are more than just lines of code; they are vibrant ecosystems where diverse individuals come together to share ideas, inspire one another, and drive innovation. The Apache Iceberg project exemplifies the remarkable strength of community collaboration. Since its inception, the project has attracted an array of talented developers…Apache Iceberg4 min readApache Iceberg4 min read
Apr 25Building a Streamlit app on a Lakehouse using Apache Iceberg & DuckDBDeveloping full-stack data applications can be a massive overhead for data scientists or analysts who would need to spend a considerable amount of time building intuitive interfaces and managing the design aspects & infrastructure of such applications. This also removes their focus from building core data-specific features like ML models…6 min read6 min read
Jan 10Reflecting on the Year 2022A Very Happy New Year to all the readers! 🎉🎉 To say that 2022 has been an exciting & thrilling ride will be kind of an understatement. In fact, this was one of those times when I took on a couple challenges to decide what mattered most to me in…Data Engineering7 min readData Engineering7 min read
Oct 25, 2022Subsurface 2023: The open lakehouse conferenceThe Subsurface LIVE conference is back. It is the industry’s premier cloud data lake conference, featuring some of the most exciting, innovative speakers and open-source projects propelling today’s cloud data lake ecosystem. The call for speakers is open until 1st November, and we’re looking for members of the data lake…Data Lakehouse5 min readData Lakehouse5 min read
Aug 8, 2022Building a Tableau dashboard directly on the data lake with DremioAs a BI engineer, you are tasked with developing a new dashboard for the revenue unit of your organization. You have all the wireframes defined, had discussions with the business stakeholders to understand the key metrics, and all you need is access to the data to start working on it. …Tableau6 min readTableau6 min read
Jun 11, 2022D3.js for Data visualizationIn my last blog, I shared some perspectives on whether you should code or use off-the-shelf tools such as Tableau for Data visualization. If you are interested to read it, below is the article. Data visualization —tools or code? Effective data visualization is paramount to any organization in today’s data-centric world. The ability to derive…dipankar-tnt.medium.com It is a no-brainer that if our motivation is building a custom or completely new visual…D3js6 min readD3js6 min read
Jun 9, 2022Apache Iceberg — Top 5 featuresWith the rise of the cloud data lakes to democratize data & enable all sort of analytical workloads(BI, Data Science), one of the most critical decisions to make for an organization’s data architecture is deciding which table format to adopt. If you are new to the space of data lakes…Data Lake5 min readData Lake5 min read
Published inMLearning.ai·Jun 7, 2022Data visualization —tools or code?Effective data visualization is paramount to any organization in today’s data-centric world. The ability to derive intricate patterns from data and explain them using visual metaphors has given new directions to data visualization. …Data Visualization5 min readData Visualization5 min read
Published inData Engineer Things·Jun 3, 2022Demystifying Table Formats in Data Lake (feat. Apache Iceberg)In my early days of learning & sharing my experience with the data community about the Apache Iceberg table format, I came across a couple of ‘need-to-know’ questions from the community. So, I thought it would be great to go back to some basics. This will also serve as a…Data Lake5 min readData Lake5 min read
May 9, 2022My first 2 weeks at DremioAs I wrapped up my 2nd week at Dremio, I wanted to reflect on a few thoughts as I embark on this new journey into the world of large-scale data analytics/engineering from Visual Analytics(Qlik, BI, ML) space. Some of the ‘key’ terms that I will have my focus on. So…Data Engineering7 min readData Engineering7 min read
