- Видео 10
- Просмотров 1 623
Aymane Maghouti
Добавлен 15 июл 2018
Welcome to my channel,
I'm a data and software engineering student passionate about technology, and this channel is where I share demonstrations of my projects in:
Software Engineering: Check out my latest software development projects, coding practices, and more.
Data and Big Data Engineering: Explore my work with data processing, analysis, and big data technologies.
Machine Learning: See my experiments with machine learning models, algorithms, and applications.
Join me on my learning journey as I tackle various challenges and showcase what I’ve learned. If you're interested in these fields or are on a similar path, I hope you find my projects insightful and inspiring!
I'm a data and software engineering student passionate about technology, and this channel is where I share demonstrations of my projects in:
Software Engineering: Check out my latest software development projects, coding practices, and more.
Data and Big Data Engineering: Explore my work with data processing, analysis, and big data technologies.
Machine Learning: See my experiments with machine learning models, algorithms, and applications.
Join me on my learning journey as I tackle various challenges and showcase what I’ve learned. If you're interested in these fields or are on a similar path, I hope you find my projects insightful and inspiring!
Real-Time Data Streaming, Processing with Kafka, Spark & Debezium : Event-Driven Architecture
In today's fast-paced world, real-time data processing is becoming more crucial across various industries. Whether it's financial transactions, social media monitoring, or tracking changes in large datasets, we often require tools that can capture changes in data automatically without any human intervention. This is where event-driven architectures come into play, offering scalable and flexible solutions to handle data streams efficiently,. That’s where Change Data Capture (CDC) mecanism comes in, and recently I had the chance to explore one of the most interesting CDC tools: Debezium.
👉 Debezium, as I've come to understand, listens to your data source for changes (insertions, updates, de...
👉 Debezium, as I've come to understand, listens to your data source for changes (insertions, updates, de...
Просмотров: 86
Видео
Cloud Data Engineering Project : Migrating an On-Premises Data Pipeline to AWS
Просмотров 2313 месяца назад
In this video, you will see: - Introduction: Overview of the project and the motivation behind migrating from an on-premises setup to AWS. - On-Premises Pipeline Overview: Understanding the initial pipeline using HDFS, Spark, Kafka, PostgreSQL, and Power BI... ( details here : [ ruclips.net/video/iClZyC_TZyA/видео.htmlsi=YXKFSugQ-rqXulUJ ]) - AWS Pipeline Architecture: - Data Collection: Scrapi...
Exam Planning System With Spring : Part 3 - Deployment with AWS
Просмотров 854 месяца назад
After developing a web application using the Spring framework. Now, it's time to deploy it on the cloud using AWS. This involves transitioning from a traditional on-premises setup to a modern cloud-based infrastructure, moving our application code and database to cloud services. 🌐 Deployment Architecture on AWS: AWS Elastic Beanstalk 🚀: This Platform as a Service (PaaS) simplifies the deploymen...
Big Data Project : Patent Analysis - IA in Education
Просмотров 894 месяца назад
Here is an overview of the project architecture and idea: Part 1: Data Collection - Sources: We identified Google Patents, OPS, and UPSTO as our primary sources. - Methods: We used APIs and web scraping techniques to collect data. Part 2: Data Storage - We stored the data in MongoDB Atlas (cloud) and HDFS (on-premise) as Data Lake Solutions. Part 3: Data Transformation - Using the Medallion arc...
Exam Planning System With Spring : Part 2 - Technical Demonstration
Просмотров 1074 месяца назад
This project leverages a layered architecture to ensure maintainability and reusability by separating business logic, database access, and presentation logic. Our application is structured into four main layers: - Business Object (BO) Layer: Contains persistent entities. - Repository Layer (DAO): Manages data access. - Service Layer: Contains business logic. - Controller Layer: Handles presenta...
Exam Planning System With Spring : Part 1 - Overview
Просмотров 514 месяца назад
This project leverages a layered architecture to ensure maintainability and reusability by separating business logic, database access, and presentation logic. Our application is structured into four main layers: - Business Object (BO) Layer: Contains persistent entities. - Repository Layer (DAO): Manages data access. - Service Layer: Contains business logic. - Controller Layer: Handles presenta...
Big Data engineering Project : Smartphone Price Prediction in Big Data Environment
Просмотров 3166 месяцев назад
In the realm of big data systems, leveraging robust architectures is key to effectively managing and analyzing vast amounts of data. After delving into various architectures and solutions, including Kappa, Lambda, SMACK, and the Hadoop ecosystem, I was interested by the versatility and power of the Lambda architecture. Lambda, as I've come to understand, is a sophisticated big data architecture...
Real-time data pipeline using kafka
Просмотров 4156 месяцев назад
🔍 The Journey: From Data Collection to Real-Time Insights 1️⃣ Data Collection: Metrics data is collected from the local computer using the psutil Python library. 2️⃣ Kafka Producer:The collected data is sent as messages to Kafka topics through a Kafka producer. 3️⃣ Real-Time Consumption: Kafka consumer read the messages from the topics, process them, and load the data into SQL Server DataBase. ...
Data Engineering Project using Hadoop Ecosystem
Просмотров 1376 месяцев назад
🌐 Project Overview: This project demonstrates the process of extracting data from a MySQL database, transferring it using Apache Sqoop in batch mode, storing it in Hive Data Warehouse (the data is actually stored in Hadoop Distributed File System (HDFS)), and performing analysis using Hive Query Language (Hive QL) (similar to SQL). The data is then visualized in Power BI after connecting the Hi...
Machine Learning project : Sentiment Analysis for Jumia Reviews & Smartphone Price Prediction System
Просмотров 1319 месяцев назад
🌐 Sentiment Analysis and Recommendation System: - Leveraging sentiment analysis algorithms, I collected and analyzed customer comments from Jumia using BeautifulSoup. The system categorizes sentiments as positive or negative and recommends or advises against a product accordingly. For a hands-on experience, I've also simulated this functionality through a user-friendly web application using Fla...
0:00 - Solution Architecture 1:26 - Debezium Demo 3:20 - Spark Streaming Process 5:24 - Dashboard App [Spring boot - React] 6:23 - Notification service 7:33 - Dashboard Exploration 9:31 - Recap of the project
can you send me this project source code please, i need it for my pfe
To see my latest project refer to : [ ruclips.net/video/MDplJJlo-y8/видео.htmlsi=yTQbgnnUXjEv7FEH ]
Exam Planning System With Spring : Part 2 - Technical Demonstration : ruclips.net/video/S7ZgZ5GiOhM/видео.htmlsi=EKNUxSsiDF-c6qtX
Part 1 : ruclips.net/video/0VPMqNcFX-M/видео.htmlsi=VU7ZpnXNDCPWh49l
Part 2 : ruclips.net/video/S7ZgZ5GiOhM/видео.htmlsi=hiXyV19i3F349aDk
bravo
Merci
@@aymanemaghouti2065 can you maybe give a tutorial on how to set up the evironment(requirements). For completely newbie.
@@aymanemaghouti2065 are you planning on making a tutorial on setting up the environment or requirements? For complete newbie ☺
@@ChenyangZhang-si4wi For now no because I am working on other projects, but after yeah I can upload a tuto where I will explain how to set up the infrastructure including : Hadoop installation Kafka Spark and pySpark Airflow with docker Postgre db Hbase Power bi desktop and how to connect with postgre SQL in direct mode How to develop a spring boot application ( maybe flask also ) Otherwise if you want to run the project, reach me out, this is my email aymanemaghouti@gmail.com
(For detailed insights into the xgboost model, refer to [ www.linkedin.com/posts/aymane-maghouti_datascience-sentimentanalysis-priceprediction-activity-7155616601214447617-VKr1? ]) Curious to explore further? Dive into the project's GitHub repository [github.com/aymane-maghouti/Big-Data-Project]
🔗 Link to GitHub Repository: I hope you find it inspiring and clear, feel free to explore it here : github.com/aymane-maghouti/Sentiment-Analysis-for-Jumia-Reviews-and-Smartphone-Price-Prediction-System 🔍 See my latest project here : aymane-maghouti.github.io/
github repo : github.com/aymane-maghouti/Real-Time-Data-Pipeline-Using-Kafka contact me in LinkedIn : www.linkedin.com/in/aymane-maghouti/
🔗 Link to GitHub Repository: I hope you find it inspiring and clear, feel free to explore it here : github.com/aymane-maghouti/Mobile-Data-Hive-Insights contact me in LinkedIn : www.linkedin.com/in/aymane-maghouti/