106.Databricks|Pyspark|Automation|Real Time Project:DataType Issue When Writing to Azure Synapse/SQL

[@abhishekrajoriya2507]

Thanks a lot sir for sharing such real-time automation scenarios, this will help us to make more dynamic solutions

[@rajasdataengineering7585]

Happy to help

[@ramswaroop1520]

in one of the interview they asked, "What is biggest/wellknown drawback in azure synapse analytics ", can you please clarify.

[@chappasiva]

Hi Raja,
Nice explanation.
where can we get the note books. please provide me those.

[@VenkatGolivi]

Excellent you deserve for more subscriptions

[@rajasdataengineering7585]

Thanks

[@Ramakrishna410]

Hi Nice explanation. My requirement is source is CSV, target is delta table.. so how to fetch column name and datatype for delta table..

[@sravankumar1767]

Same logic how can we apply Mongo db and cosmos db, can you please explain. Basically these databases support different datatype right ? Can you please make one video on this one 🙏

[@rajunaik8803]

Hi Raja, I guess we can also do type casting according to target system before appending! Do you see any challenges in type casting?

[@rajasdataengineering7585]

Hi Raju, data type is different between azure DW and spark so created this automation solution

[@naren06938]

I hope it is Database, not warehouse....

[@sravankumar1767]

I have one doubt how can we select query for azure Mongo db

[@suvratrai8873]

what if there are more columns in dataframe, compared to table?

[@rajasdataengineering7585]

We need to apply workaround of dropping unwanted columns from the dataframe

[@suvratrai8873]

Then maybe we can add that condition too right, just to cover all possible scenarios?