I would configure the expectation results storage location to be a bucket and then have a pipeline that takes the expectation results and stores them in a database
Hi - Thanks for sharing this video. When I tried running it, I kept getting errors like "raise gx_exceptions.DataContextError( great_expectations.exceptions.exceptions.DataContextError: expectation_suite strawberry_suite not found". During a deep dive into the code, i found there is a condition "self.expectations_store.has_key(key)". I am missing something concerning the store.?
I've only recently started using Soda so I'm not sure if I have enough experience to form a definitive opinion, but I have definitely enjoyed the UX much more so far, SCL is a lot more human readable than great expectation "expectations" imo
May I know if, we can create our own expectations. If I have a expectations who is not in the script that is on the documentation Could I create my own one?For Example, if one column is created from a formula and used a diferent database Could I create a expectation of who makes the math? Hi, from Colombia :)
Very simple video with excellent explanation and not overcomplicating things. Thanks for sharing it!
Thanks for watching!
Great explaination!Any thoughts on how we can save the great expectation results in the Database?
I would configure the expectation results storage location to be a bucket and then have a pipeline that takes the expectation results and stores them in a database
Very useful bud. Thank you.
No problem, do it all for you!
@@thedataguygeorge will reach out on the side to talk shop if that's cool, loving your work.
Always cool!
Hi - Thanks for sharing this video. When I tried running it, I kept getting errors like "raise gx_exceptions.DataContextError(
great_expectations.exceptions.exceptions.DataContextError: expectation_suite strawberry_suite not found". During a deep dive into the code, i found there is a condition "self.expectations_store.has_key(key)". I am missing something concerning the store.?
On failure of great expectation validation, would this raise alerts?
Yes as long as you have Alerts configured for your Airflow DAG
Also how can we automate the threshold changes with the changing business needs?
You'd want to have another helper pipeline that checks for changing business requirements and then either alerts you or makes adjustments
Thank You
No worries, let me know if there's any other videos you'd like to see!
Hi, do you prefer soda or great expectations?
I've only recently started using Soda so I'm not sure if I have enough experience to form a definitive opinion, but I have definitely enjoyed the UX much more so far, SCL is a lot more human readable than great expectation "expectations" imo
Also can we include more than one expectation in the expectation file?
Yes!
On minute 3:57 how could create the default file of great_expectations? I created the json but I got a blank one.
You then just fill out that json with all the expectation info you want!
May I know if, we can create our own expectations. If I have a expectations who is not in the script that is on the documentation Could I create my own one?For Example, if one column is created from a formula and used a diferent database Could I create a expectation of who makes the math?
Hi, from Colombia :)
Definitely can create your own expectations, honestly one of the best features of great expectations!
@@thedataguygeorge Do you have a video-tutorial where you’re teaching how to connect GX with Databricks? 😊