Building a Scalable Record Linkage System with Apache Spark, Python 3, and Machine Learning

Nicholas Chammas, author of Flintrock, and Edward Pantridge data scientist and artificial intelligence researcher discuss the MassMutual has hundreds of millions of customer records scattered across many systems. There is no easy way to link a given customer’s information across all these systems to build a comprehensive customer profile. Building such a profile has important applications in many areas of MassMutual’s business, from marketing to underwriting.
Learn more here: databricks.com/session/buildi...
Article you might like: databricks.com/session/levera...
About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Read more here: databricks.com/product/unifie...
Connect with us:
Website: databricks.com
Facebook: / databricksinc
Twitter: / databricks
LinkedIn: / databricks
Instagram: / databricksinc Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. databricks.com/databricks-nam...

[@damaya1982]

Thank you for this video! This is a superb contribution to a field that has surprisingly little documentation outside of academic papers.

[@hope21singh]

Really good video, I am fresh to Records Linkage with no technical experience but it made sense to me.

[@mariamzayed91]

omg thank you!

[@youssefboukhadmi3032]

thanks

[@acsrr4288]

any chance there is some sample codes for reference?

[@stevanmeandzija]

Hi, how is this scaled, on lets say 10 million records?

[@harishkumarthirunagari8457]

Hey, i am also having same issue scaling record linkage to millions of data in CRM. Did u find any way to do it ?