Great overview! Gave me some nice ideas to focus on! Say you are building datapipelines for e-commerce, do you have any tips on what to watch out for in terms of GDPR? Is it ok to store e-mail adresses and physical adresses in a data-warehouse, or should these immediately be anonymized? If so, where in the pipeline would you anonomize this data? Thanks for your videos!
#1 concern should be making sure that you implement encryption and other proper security controls throughout the entire pipeline, don't necessarily need to anonymize the data as long as you have it stored in a secure location and can delete it upon customer request.
I just started a job working with pipelines and this stuff is golden! Thank you!!
No problem, glad you find it helpful!
Nice insightful video :-) Be aware of the ghost of overengineering
Great overview! Gave me some nice ideas to focus on! Say you are building datapipelines for e-commerce, do you have any tips on what to watch out for in terms of GDPR? Is it ok to store e-mail adresses and physical adresses in a data-warehouse, or should these immediately be anonymized? If so, where in the pipeline would you anonomize this data? Thanks for your videos!
#1 concern should be making sure that you implement encryption and other proper security controls throughout the entire pipeline, don't necessarily need to anonymize the data as long as you have it stored in a secure location and can delete it upon customer request.