Import Data and Analyze with MATLAB
HTML-код
- Опубликовано: 22 авг 2024
- Data are frequently available in text file format. This tutorial reviews how to import data, create trends and custom calculations, and then export the data in text file format from MATLAB. Source code is available from apmonitor.com/c...
You are a wonderful teacher, 8 years and countless updates later yet your tutorial still stands!
thank you sir.. its because of you i am able to complete my final year project.
From where we can get a data file
@Bushra Noor-ul-ain i am getting error can you help
You sir saved my behind on an Engineering lab, thank you!
🤣Glad to hear that it helped!
Thanks a lot for the easiest solution of the problem! From India!
thank you very much i was facing the problem since 2 days,now its solved
How Can i download data of ball bearings ( base de données de roulement )
Maybe a Google search?
Je ne trouve pas la base de données de roulement
@@mezourachahinez5721 Does this help? www.ntnamericas.com/en/website/documents/brochures-and-literature/catalogs/ntn_a1000xi_ball_and_roller_bearings_lowres.pdf
Thanks for this video. Thank you for being brief and clear.
Thank You very much for brief and simple presentation
I am using a Benewake LIDAR TF mini plus and it is working perfectly via UART but I want to use it using TTL converters. I used Termite and the data is unreadable, can you guide me a little on what is the best way to use it with TTL converters?
What type of TTL converter are you trying to use? Here is something I found online: "UARTs transmit one bit at a time at a specified data rate (i.e. 9600bps, 115200bps, etc.). This method of serial communication is sometimes referred to as TTL serial (transistor-transistor logic). Serial communication at a TTL level will always remain between the limits of 0V and Vcc, which is often 5V or 3.3V."
I was trying to plot the data but it said there was an error in color/linetype, I pretty much copied exactly what you did to get a better feel of matlab, so I'm not sure why it was not working.
Try copying the code from here: apmonitor.com/che263/index.php/Main/MatlabPlots specifically at apmonitor.com/che263/index.php/Main/MatlabPlots?action=sourceblock&num=1 or apmonitor.com/che263/index.php/Main/MatlabPlots?action=sourceblock&num=2
You can get the source files for this problem from: apmonitor.com/che263/uploads/Main/matlab_data_analysis.zip
i want to put data in matlab automatically which i collect them from sensor and at a time by using this data i want to creat a running plot , that means automatically input in matlab and creat a running plot ,sensors will give me data continually and always it will be uploaded in matlab and at a time creating a plot which is running....
how can i do this..?
There is an example here: apmonitor.com/heat.htm
You saved my life
sir, I have 200 x,y coordinates, can i generate a matrix in matlab using those coordinates? from that matrix again I want to generate an Image, Is that possible in Matlab??
Try a meshgrid www.mathworks.com/help/matlab/ref/meshgrid.html I'm not sure about images
sir how to import a excel file to the matlab and how do i access the contents of the excel file in if else condition to complete my code ..please help ..sir
Here is a link that helps with Excel: www.mathworks.com/discovery/matlab-excel.html
How can we convert the raw data (802*2) where 802 is no. Of rows and 2 is no. Of columns into an image?
See the 3D Numpy arrays for information on how to do this with Python: apmonitor.com/dde/index.php/Main/PythonNumpy
sir ,if my text file contains some texts in the starting other than numericals how can I aviod that from
reading by matlab
(Error using ==> load
Number of columns on line 1 of ASCII file C:\Users\Sherin
Abraham\Downloads\mag\abg20000212dmin.txt
must be the same as previous lines.)
this is my error message
Does this help? www.mathworks.com/matlabcentral/answers/27734-how-can-i-skip-lines-when-loading-a-text-file
I am able to run my calculation when I enter in my row value as { : , 2 }, but I want to only use rows in increments of 10. I would assume this would look something like { 1:10, 2} however I keep getting an error message. Please advise! Thank you!
Try something like this: Your_array{1:10:end,2}
How to open and view the DAT.file which is in binary image format as well as table text format.
Try imopen: www.mathworks.com/help/images/ref/imopen.html
This helped me a lot. Many thanks!!!!!!!!!!!
Sir, I need to implement a similar operation. I need to take the text file, where the text file includes some set of rows for a particular date and some rows for some other dates and so on.
Now, i need to take the data corresponding to the required data (the user will enter the date) Please note that the first column of each row contains the date. The Matlab needs to compare first column of each row and if the first column data ie: date of the current row matches with the date entered , then this row will be copied to the a new test file. The code will do this process till the end of the text file, and finally, I will get the required data text file only.
Kindly response
Regards
+Ayush Sharma, you may need to use a different method such as textscan to read in the date information: www.mathworks.com/help/matlab/import_export/import-formatted-dates-and-times.html Once you import the data, you can use one of these methods to lookup the row: www.mathworks.com/matlabcentral/answers/218033-efficient-method-for-finding-index-of-closest-value-in-very-large-array-for-a-very-large-amount-of-e
I am using matlab 2011b and when I try to import my excel file like this it does not work. My excel file contain separate column as date+ time (this is one column Eg. 1/2/2013 12.30am) , Region (text) , and Power (number) . Pls help me to import my excel file to matlab.
Rasika Dilhani You probably need to select in Excel the Save As option and save the file as a txt or csv file. There is also a xlsread function in MATLAB if you'd like to read an Excel file directly: www.mathworks.com/help/matlab/ref/xlsread.html
APMonitor.com Thank you so much. Now it works properly. I waste too much my time to do this. Your guideline help me to success this matter.
thanks for sharing.. i have a 3 colomns data. i want to find radius of
curvature of this data. Could you help me please ?
Here is information on Regression with MATLAB: apmonitor.com/che263/index.php/Main/MatlabDataRegression
Thank you kind sir. You have made easy a daunting task
Thanks sir. After I load my .mat file into the editor. How can I read each variable column in my dataset?
disp(x(:,2))
could you please tell me where you got the data? It would help a lot. Thanks!
Thanks dude. It really helped me
thank s for all
Thank you sir. I have 6000-36000 values in form of rows. Each time i copy data from excel file to matlab, some of values are shown as NaN, how can i change NaN to numeric values... The data is large and i can't even put every value by hand. Is there any solution?
You can type Ctrl-h to open up the dialog box for Find and Replace.
@@apm Thank you. So I have to change each value separately?
@@muqtasidaliawan no - you can select "Replace All"
@@apm Thank you so much for your reply. It will help. 👌 👍
can you explain why i get an error like this?
Traceback (most recent call last):
File "", line 1, in
data[0:4]
NameError: name 'data' is not defined
+kisore hari It is probably because the file import wasn't successful. Take a look at the companion video to this one (on Python), starting here: ruclips.net/video/Tq6rCWPdXoQ/видео.htmlm38s This is a MATLAB tutorial but you appear to be using Python.
Quite useful! Thanks a lot
where can I download the code ? thanks
apmonitor.com/che263/index.php/Main/MatlabDataAnalysis
Thanks sir for the great explanation. Btw I really need your help, actually I got data file in .mat format which are 1 input data (4000x1) and 2 output data (4000x2). How can I use these data in order to get the transfer function model? I already used the 'iddata' command, but some how I'm not sure how to determine the number of pole and the number of zero.
Here is information on obtaining a model from data: apmonitor.com/do/index.php/Main/ModelIdentification
Hi, do you know how you would find the x values which cross a plot at y=0, really struggling!
Thanks
+Fiona Cameron, you can do a linear regression (apmonitor.com/che263/index.php/Main/MatlabDataRegression) and then set y=0 and solve for x in the equation for a line: y=slope x + intercept.
Sir, can you please suggest any project that I can start in ML!?
Here is an example to help get you started: apmonitor.com/do/index.php/Main/DeepLearning
Oh soo much Thank you!!!
great video. Thank you sir.
sir, suppose i have single column of 16 values, if i want to represent these values as 4x4 matrix, then how to do it..?
I'd recommend the reshape function: www.mathworks.com/help/matlab/ref/reshape.html
Thanks for showing us how we can analyse data with Matlab ! I would appreciate if you give me an explanation about how we can import data online from an SQL server ! Thanks
+Zied Nouir Mahjoub I don't have a video on SQL statements but here is some information online: www.mathworks.com/help/database/run-sql-query.html and www.mathworks.com/help/database/ug/create-a-query-using-a-matlab-variable.html
I found this tutorial very interesting, i really need your help on how to import and excel data into matlab and develop a linear, non linear regression model and ANN (Artifial neural network) with the imported data. Thank you looking forward fro your responce
+ABDUL ALIM IBRAHIM I don't have a tutorial on ANNs specifically but here are some on linear and nonlinear regression: apmonitor.com/che263/index.php/Main/MatlabDataRegression
Am doing my project on import analysis so that I need find reasons for fluctuations in imports ... Whether I can find using multiple regression
There are multiple courses that teach regression concepts (basic to advanced) here: apm.byu.edu I recommend starting with this material: apmonitor.com/che263/index.php/Main/MatlabDataRegression
Nice work sir!! Can you tell me how Can i import striffness Matrix in MATLAB , when i type import('Matk.txt') i have all the Matrix parameters in the same column!!!
Try the reshape function: www.mathworks.com/help/matlab/ref/reshape.html
Thanks, very useful to me
Dear Sir, its work with tracefile of NS2 please ?
You can call a Python function from Matlab. Try this package: pypi.org/project/traceanalyzer/
Thank you very much for such a useful and neat tutorial.
Thanks!
How can we do that using MATLAB guide??
Here is information on importing data with a GUI: www.mathworks.com/matlabcentral/answers/68470-import-data-using-guis
How can I open huge .nc data like this one
I'm not familiar with .nc files. If they are text files then it shouldn't be a problem.
thx ….you saved my file
I'm glad it worked for you.
thanks for the info! how can I implement a search feature that would bring up the value I want to associate with each name given (say, gene accession number with the name of a gene). I'm sorry if this is very simple, but I'm brand new to programming!
+ray ho, look at functions such as strfind and strcmp. Once you find the name, you can look up the associated value or use the index to go onto the next step.
hello, professor, I would like to simulate human detection by hog and Inriaperson dataset .my dataset contains train and test part and in each part have pos and neg what should I do?? how should I load this huge dataset??
I'd recommend loading each file individually and then perform the training and testing / validation. Alternatively, you can load one file and then parse out the training and test data. Here is a related example but in Python: apmonitor.com/do/index.php/Main/DeepLearning
thanks for sharing your work
sir,
when i plotting there is an error occurs,
"Vectors must be the same length."
ps:
i took 6 values of data with the all column of time.
Islam, you may need to transpose one of your data vectors so that you have the same number of rows. You can see the size of each thing you are trying to plot with the "size" function: www.mathworks.com/help/matlab/ref/size.html. Make sure that x and y have the same dimensions. You can use the command "hold on" to plot multiple data sets on the same plot.
Try adding an ‘ to the vector. It’s probably because one is a column and one is a vector
How to handle a time series data with occasional data gaps? for example how to make a weekly average of monthly data file, when data is missing...a loop will land into errors ...please help
+soumendra singh One way to do this is to indicate the missing data as NaN values. Then you can apply the nanmean function to average over the data, ignoring the NaN data (www.mathworks.com/help/stats/nanmean.html).
If you don't know if a particular number is missing or not and want to do this as a loop, then use the isnan function to determine which elements are numbers (www.mathworks.com/help/matlab/ref/isnan.html)
sir when i try to load the data it gives me the error
You can get the program and data file from apmonitor.com/che263/index.php/Main/MatlabDataAnalysis
Thank You Sir ☺
It is an enterprising video for me. But I have a separate problem using data collected to detect malware. How can I input this series of data to SVM in order to generate a result?
Bashir Muhammad Take a look at the SVM documentation here: www.mathworks.com/help/stats/svmtrain.html It appears that there is a particular input format that is required.
hi.i am very new to matlab my problem is how to divide large amount of data into 4 group the data contain 181*500 .please i need your help
+Momen Abdul Razzaq You could do it with the following commands:
group1 = data(1:end, 1:100);
group2 = data(1:end, 101:200);
group3 = data(1:end, 201:300);
group4 = data(1:end, 301:500);
Just adjust the row or column ranges to select the appropriate data. You can also shorten the part "1:end" to just ":".
sir if i had two different sensors data which are sampled at different rate and i want to plot in single graph i want calculate difference??
+Saeed Muhammad, you have a couple different options. (1) create an interpolation of your data and compare the functions that fit your data (see apmonitor.com/che263/index.php/Main/MatlabDataRegression or www.mathworks.com/help/matlab/ref/spline.html). (2) compare nearest data points by finding one value that is close to the time in the other list (see stackoverflow.com/questions/8089031/how-do-i-find-values-close-to-a-given-value)
thanks allot sir for yours reply
Sir I m using matlab online how can extract my excel file
Does it work to upload your Excel file to drive.matlab.com so that you can import it?
Bardzo dobre :) polecam
Thank You very much :)
Great Tutorial
I want insert data to neural network as two dimensional ,so for example i have two group T and B .T have one raw and 10 column and B also same , so i want both of this data become as one input but 2-Dimensional and target also it will be same process .my question are neural network accept (2-D input , 2-D target) and if it's accept how i can do this process in matlab ,thank you for your help.
Alaan, check out the ANN (artificial neural network) toolbox. It will help you fit a model. However, you may find that an ANN model extrapolates poorly. With such a small system (2 x 2) you may find other forms that extrapolate better such as a first principles model or better empirical approaches. Make sure you keep a separate data set for validation.
Hello sir, I have a problem with importing data.
Let's say I have a 5 set of data. Each data contain 50 values. so how I am gonna do to create a benchmark and combine it in a workspace for each set?
After you import the data files, you can combine them with combined_data
= [data1 data2 data3 data4 data5] for column oriented data or combined_data = [data1; data2; data3; data4; data5] for row oriented data.
Awesome, thanks!
Can you explain that if i want to find equation of the plot which i have plotted then how can i do that???
Here is information on regression: apmonitor.com/che263/index.php/Main/MatlabDataRegression
hello, how can I generate multivariate random data add percentage of contamination data(outliers)
+Tareq Atiany There is an example of noise, outliers, and drift for contamination of data on the dynamic optimization course web-site here: apmonitor.com/do/index.php/Main/DynamicData (see the example at the bottom).
how can I use POS tagger and SentiWord in MATLAB for opinion mining/TextMining and feature extraction??? dataset is in excel extracted by twitter..
This is easy with Python or NodeRed but I haven't tried it with MATLAB. Here is another question that you may want to follow to see if Mathworks provides an answer: www.mathworks.com/matlabcentral/answers/130100-sentiment-analysis-and-opinion-mining
hello, please help me am hopeless its my first time to work with matlab please how can i load my database on matlab (my application is to classify this data using decision trees in matlab) i put my data in text file or execl file??? and how do i load it in matlab for machine learning ??????????
+toukal manel A text file is probably the easiest method to import data. You can also access the database directly: www.mathworks.com/products/database
I am a first time user of matlab and am trying to use Matlab to analyse torque at the ankel of the leg after conditioning it with some stretching protocols and using a dynamometer to measure and collect data of force at angle of 5,10,15.. How do i calculate the torque at sey 5deg?
Check out the optimization course material at apmonitor.com/me575 under the application project section. There was one project that was similar to yours at apmonitor.com/me575/uploads/Main/2013_Knee_Simulator.pdf
How to do pca using matlab
www.mathworks.com/help/stats/pca.html
How Can i import Data file into simscape multibody, thx in advance Dear
I'm not sure about that. Maybe check with mathworks?
@@apm i'm sorry for my question, it is not clarify , i extracted mass and striffness Matrix from Ansys workbench (.txt) files format and i want to import them to MATLAB, how Can i do it plz?
Hello, I have a zipfile(named proposal) that contains 3 different devices and each device has a list of voltages from (0 volts - 10 volts) and each of these voltages has a very long list of currents(numbers only), how can I use the data from all 3 devices(device1, 2 & 3) to calculate the Autocorrelation for each signal, calculate the Fourier Transform of the Autocorrelation for each signal and the Fast Fourier Transform for each signal. With time = 10^-15.
Everything I tried hasn't worked. Thanks.
I recommend that you try this function for autocorrelation: www.mathworks.com/help/econ/autocorr.html I also like to use the svd function to determine the degree of correlation between the signals. The relative magnitude of the singular values is the thing that you want to use to determine if the signals are unique versus just scaled values of each other. Here is help on the fft function: www.mathworks.com/help/matlab/ref/fft.html
Hello sir , i use import tab from the Toolstrip , but my file contains both texts and numbers so i have to customize the selection by A , B , C ,... rows , is there a way to save this customized settings so i dont have to do it all over again every time i import data from these text files , as those texts files all have the same format so it will be easier if i could save my selection settings , Thank you
+Youssef Zakaria, one way to avoid this is to import the data as a cell array and then you don't need to differentiate between numbers and text. www.mathworks.com/help/matlab/cell-arrays.html
Sir what if i want to process the data in time attribute.Like time has a set of data ,in which if anyone of the data is higher than some threshold value print some information.
This may help: www.mathworks.com/matlabcentral/answers/29099-importing-formated-time-data-into-matlab
Slick demonstration.
Thanks!
matlab solution f(x)=3cos(1.7x)e 0.3x 2sin(1.4x)e0.3x for- 7
Here are suggestions on solving equations in MATLAB: apmonitor.com/che263/index.php/Main/MatlabSolveEquations
hello! i've been trying to import some special functions that my teacher gave me to practice Matlab, but i've been struggling to load the functions into my workspace. The error says that ""Number of columns on line 1 of ASCII file (my file directory) must be the same as previous lines."", and i feel like i am loading the functions badly because Matlab thinks they're supposed to be ASCII files when they're not: how do you think i could fix this? is that the actual problem or is it something else?
thanks in advance, have a nice day!
+BlinkTF2, for functions you should use the addpath command. Place your functions in a directory and then include something like addpath('c:/my_functions'). You should then be able to use the functions in that directory.
what if it is in .dat file format?
As long as it is text, you should be able to import it. Here is information on changing file extensions: ruclips.net/video/YlN9ax7yC0A/видео.html
very good video but why does he sound like Mr. Garrison from south park
That's a new one that I haven't heard before. 😀
I don't know why loading isn't working for me ; u did it with text file let me see if that works ..
What type of file are you trying to load?
i have 1x1099 double value in 'sg' variable max value 39.2885 and min 30.1231 how i can normalized it b/w 0 and 1 can u please help me with that i will be thankful
You could normalize with something like this: x = 100.*randn(1,10); x_norm = (x - min(x)) / (max(x) - min(x))
How to creat data file?
You can get the data file from apmonitor.com/che263/index.php/Main/MatlabDataAnalysis An easy way to create your own is to use Excel and export as a CSV file.
Thank you for this wonderful video Sir. I would appreciate it if you help me out with the following problem solution. After loading the dataset I want to select only the rows in which Response time is 3 cost is
You could create a loop and select only those rows that have that condition. Alternatively, it is easy to do this with Python (Pandas package): apmonitor.com/pds/index.php/Main/CleanseData
How I can convert coe file ( generated by ip core in fpga) to image file by using matlab??
Sorry, I don't have experience with COE files.
I am very new to matlab, my difficulties are how to import data from excel to matlab and running a code to do analysis like PCA ,LDA etc.please I need your help....thanks.
+Ibrahm Musa Here is a link on LDA: www.mathworks.com/matlabcentral/fileexchange/29673-lda--linear-discriminant-analysis and PCA: www.mathworks.com/help/stats/pca.html
One other resource that may help you as a beginner is to visit the Matlab section of my course on computational tools apmonitor.com/che263 Please navigate to the Matlab section for introductory videos.
Thank you but how to select aset of primary dependent variable and plting four graohes of different secondary dependent variable derived from the set of primary dependent variable
hamza albazr One thing that I didn't show in the tutorial is how to create an inline function. Here is one example of plotting derived variables where the dependent variables may not have been known before the function was defined.
y = @(t) t.^2; % create a function y(t) = t^2
t = linspace(0,10); % create a vector t with 100 points between 0 and 10
plot(t,y(t)); % create a plot with t and y(t)
In the case of the tutorial video, you could select a sub-set of the times (first 5 rows) with time = sensors(1:5,1), s1 = sensors(1:5,2), s2 = sensors(1:5,3), s3 = sensors(1:5,4) and then an average with avg = @(v1,v2,v3) (v1+v2,v3)/3. You should be able to plot with the command plot(time,avg(s1,s2,s3)). Please let me know if this doesn't answer your question or if you have another question.
Hie Sir, Can I have your email address ? My question to you is, if I need to know the position of sensor 2, row 5. What will be the command for that ?
It is: sensors(5,2)
In the medical field how to create a data database for the disease
+طارق عطية There are some great resources in MATLAB for disease modeling and parameter estimation from data. Check out xps.apmonitor.com/wiki/index.php/Main/SBML for some of our recent work with the Scientific Biology Markup Language. You may also want to look at other model forms from your data such asLewis, N.R., Hedengren, J.D., Haseltine, E.L., Hybrid Dynamic Optimization Methods for Systems Biology with Efficient Sensitivities, Special Issue on Algorithms and Applications in Dynamic Optimization, Processes, 2015, 3(3), 701-729; doi:10.3390/pr3030701
Thank you sir
thank you sir . but i want to know how can i make the ANN takes the output of GLCM and make a database for ANN
I recommend that you take a look at the ANN toolbox in MATLAB and the commands related to graycomatrix.
okay thanks sir
Time to finish that report now 🤣
i have excel file and i want use data in kalmen filter code in measurement matrix from script how can i do it
+Asif Nizamani, Here is an example Kalman filter script: www.mathworks.com/matlabcentral/fileexchange/5377-learning-the-kalman-filter
I will try and update
It was not worked when tried to import It have not showed any xlsx file only mat files shown
TQ so much, I am calculating ATC Power Flow system, my code blocked given an indication related mpoption.m function, could u please lead me to right solution , TQ
+nor amira abu bakar I'm not familiar with your code. Maybe a paper like ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6559645 can help you.
I am using matlab 2011b I add to spm in to the matlab but spm not run give me error constatntly error : "undefined function file_array for input arguments of type char"
I changed file name this time matlab didnt find spm and I delete all spm file , download and back to add but problem still constient
How can I check this problem?
I'm not familiar with the SPM package - is it for analysis of brain images? It appears that the versions available here www.mathworks.com/matlabcentral/linkexchange/links/977 have newer versions (SPM8 or SPM12) - maybe try the newer versions.
APMonitor.com thank you for attention and for helpful videos ☺️
Sir, my dataset contains 6 columns and 19 rows in CSV format. But whenever I import it to MATLAB it will turn the rows to columns and vice versa. How can I make it to read the data? Thank you Sir.
you can transpose the data after importing
data = data'
Thank you Sir. If I have further question, I can still ask.
4:38 avg=mean(sensors,2); what is the 1st dimension then?
The first dimension is the row dimension (sampling instances in time for this case) and the second dimension is the column dimension (redundant sensors at one particular time point). We are taking the mean of the sensors at one particular time point with mean(sensors,2).
Sooo helpful!!
Sir, is there any way by which I can use or export the plotted graph as a signal in Simulink?
Mehfooz Ali, here is some help on importing data into Simulink: apmonitor.com/che436/index.php/Main/ProjectProgramming (see the first example).
Really well scripted. Thanks
Thank you!
Hello, I am doing ANN, because of Mu amount data training Process will stop after 2 epoch only. how I can deactivate Mu or other training parameters? Thanks
+Far id This video doesn't deal with ANNs so I'm a little surprised that you asked it here. It sounds like you need to adjust the tolerance for your solver. Are you using the ANN toolbox?
+APMonitor.com You are right, sorry about differences. I need a syntax or
comment to add on ‘advanced script’ of ANN toolbox that totally deactivate a desired training parameter like Mu. Thanks again
+Far id No problem - best of luck with your efforts.
Hello sir,
I have a problem with multiplication as I have to multiply a=101x1 matrix with b=202x2 matrix in the form of a(1,:)*b(1:2,:) a(2,:)*b(3:4,:) and so on
for that I have created 4 loops
for n = 1:1:202
for l = 0:1:100
for m = 0:1:100
for k = 1:1:101
mm(n,:) = a(k,:)*b((l+1)+m:(l+2)+m,:)
but I am getting an error 'Subscripted assignment dimension mismatch.'
Could you please help me with this??
Thank you
The inner dimensions of the matrix must agree for multiplication. You can do the multiplication yourself or let MATLAB do it for you (much more efficient). You can even take a slice of a matrix for multiplication. Here is an example:
clear all; close all; clc
A = rand(101,1);
b = rand(202,2);
MM = A' * b(1:101,1);
disp(MM)
Thank you Sir.
How to multiply a complex matrix with its conjugate in matlab.
I am trying the normal function of a*a' but not getting the desired result