This was massively helpful in getting me a little more understanding of how all this stuff works. At work I’m being requested to help on some big data applications that follow somewhat of a similar data flow but given I’m not a data engineer, all this has been confusing and overwhelming. This provided me enough of the clarity I needed to help me understand my work and how it’s not all magic!
Pls always make a videos in English only na some vdos in Tamil language some pls don’t understand Tamil like me that’s why but you explain so clear and understandable that’s really helpful 😊
Very Good and important video. Can you, if possible, create a video by doing practically using a dataset in this pipeline model? It would be very helpful for us :)
Thanks for the explanation. Can you please help to clarify the below two questions? 1.Can you give details about the volume of the data involved in this project.2.Can you explain more about the location of the shell script(where inside the cluster) used to call the jar will be placed
wow man .. just wow..... the simplest explanation I have seen .. I was easily able to relate to my etl projects .. and now am very curious to start my hands-on ... i wish could have come to this video of yours earlier .... please upload more videos.... :)
This is great, even we are using similar on-prem architecture. If possible can you make a video on migrating this project to cloud.. like replacement for spark, hive, sqoop, hdfs in aws/gcp?
Beautifully explained. We can get complete overview of big data project. Bro please make a small real time pyspark project from end to end like what you have explained in the video. Really it will help us a lot bro.
Thanks alot...It is really great explanation. Could you please say can we encrypt and decrypt the data after storing it into Hive and then use it for spark transformation
Hi, Can we bring production data into dev and uat environment? Is that allowed... Please ignore, I have watched your deployment video, and got the understanding that we won't be having production data in Dev and test environments.. thanks
How can Hive be used to store Big data files.It can just store metafiles in the metastore.Actual data blocks are stored in HDFS and its mapped to hive tables.Correct me if I am wrong
Eagerly waiting for next videos. Wonderful explanation buddy. Keep rocking and please train us along with you in this beautiful startup of RUclips session. My full support for you buddy❤️❤️❤️.
hi I have 1.5 years experience on Apache nifi,what should needs to be done to get a role in data engineering profile curruntly im serving notice period. please suggest me.
For a fresher what do you suggest , getting started with data engineering or ML? In my college placement most companies are coming for ML profile but I'm more interested towards data engineering part I'm confused Can you please suggest sir?
This was massively helpful in getting me a little more understanding of how all this stuff works. At work I’m being requested to help on some big data applications that follow somewhat of a similar data flow but given I’m not a data engineer, all this has been confusing and overwhelming. This provided me enough of the clarity I needed to help me understand my work and how it’s not all magic!
Pls always make a videos in English only na some vdos in Tamil language some pls don’t understand Tamil like me that’s why but you explain so clear and understandable that’s really helpful 😊
Thanks , all these videos are available in English too 😃
Very Good and important video. Can you, if possible, create a video by doing practically using a dataset in this pipeline model? It would be very helpful for us :)
Thanks for the explanation. Can you please help to clarify the below two questions? 1.Can you give details about the volume of the data involved in this project.2.Can you explain more about the location of the shell script(where inside the cluster) used to call the jar will be placed
This is what im searching for man thank you so much 💝
wow man .. just wow..... the simplest explanation I have seen .. I was easily able to relate to my etl projects .. and now am very curious to start my hands-on ... i wish could have come to this video of yours earlier .... please upload more videos.... :)
Very clean and clear explanation.Understood very well.Waiting for others videos.
Very very very useful knowledge sharing
This is great, even we are using similar on-prem architecture. If possible can you make a video on migrating this project to cloud.. like replacement for spark, hive, sqoop, hdfs in aws/gcp?
Very informative bro… thank you so much.. request you to put few more samples of projects please
Great video! Thank you for sharing your architecture and project examples.
Beautifully explained. We can get complete overview of big data project. Bro please make a small real time pyspark project from end to end like what you have explained in the video. Really it will help us a lot bro.
This use case is helping me a lot in my current project, Your explanation cleared my so many doubts thanks buddy ! :)
Excellent information, great job.
Thanks alot...It is really great explanation. Could you please say can we encrypt and decrypt the data after storing it into Hive and then use it for spark transformation
excuse me mam have you did any project in big data and hadoop ??
Thank You for this video. Can you explain more on audit tables, error handling & frequent fall outs.
Really helpful content. Please do more of such Project explanation videos. Thanks for sharing
very informative and usefull thnq bro
Thanks for this video, help full this video for us, can you create one more real-time use-case video.
thanks bro very clear!
Thank you for the explanation. Can you please explain about data lake preparation in Big data.
Thank you Gautham, very useful
please upload more issues or challenge faced and how you resolved it?
Hi, Can we bring production data into dev and uat environment? Is that allowed...
Please ignore, I have watched your deployment video, and got the understanding that we won't be having production data in Dev and test environments.. thanks
How can Hive be used to store Big data files.It can just store metafiles in the metastore.Actual data blocks are stored in HDFS and its mapped to hive tables.Correct me if I am wrong
Yes you are right...files are stored in HDFS and the metastore db is created in derby or mysql
bring data into hive by sqoop, now hive play role is datalake ?, to my knowledge hive is a data warehouse system.
Hi Sir,
One of my interview they asked, how you manage your source code? Can you please guide me on this?
can you send link for bigdata project explained -2
Thanks a lot bro.Waiting for another video on big data projects
hi really good explaination sir
Very great session
More to come
Awesome! Could not thank you enough!
Eagerly waiting for next videos. Wonderful explanation buddy. Keep rocking and please train us along with you in this beautiful startup of RUclips session. My full support for you buddy❤️❤️❤️.
Sure and thanks for your support and love
Thanks it helps alot
hi how much scope for hadoop now and in future
Thanks a lot 🙂
Very helpful information buddy.
Big thanks Sir...!
You mentioned sqoop used to transfer data from rdbms to hive or hive to rdbms right ?..db2 is also rdbms ??
Yes, he means IBM DB2...
Thank you!
is there any sites that has these sort of data
hi I have 1.5 years experience on Apache nifi,what should needs to be done to get a role in data engineering profile curruntly im serving notice period. please suggest me.
how do i get this sample data... i want to implement simple project
Good work. Thanks for sharing.
Hi sir how to connect with you if we are having any doubts...
where to find the data sets to build the project shown in the video
Good one. Thanks a lot!
Good work bro👍Jhakaas !!
Helpful content
For a fresher what do you suggest , getting started with data engineering or ML?
In my college placement most companies are coming for ML profile but I'm more interested towards data engineering part
I'm confused
Can you please suggest sir?
please go for data engineering roles , because of MR it has less scope for now in industry ( thats the fact actually )
Thanks Sir
Hi bro do you provide any training or support?, Please revert
Will hive stores data
It is great
great
can you please this source code
please please it is very very important for me
is it better to do masters on big data i am at my btech 3rd year sir plz advice
Btech is wide enough for getting into big data , masters not req but if you are interested you can do .
great explanation
thank you sir