How do I read raw text from a webpage (ESPN Fantasy Baseball)?
5 views (last 30 days)
Show older comments
I am trying to read the raw text from the following url (using Google Chrome, and R2016b MATLAB):
The format of the text is JSON. All I need is the raw text so I can parse it with JSONDECODE. Right now I'm having to manually copy and paste the text from the webpage into a text document, which allows me to read the data as a string. I don't want to do this. I am open to any methods that will allow me to go to the above url and directly read all of the text into a string.
Bonus problem: the above url is for a public website, however I would like to be able to log into ESPN at the following url: http://www.espn.com/login then proceed with getting the text from the url.
To sum up, I want to do the following:
1) Log into ESPN (go to its url and input the user/password, either manually or programatically)
2) Go to an ESPN url to read in its data (either the raw text which I can decode with JSONDECODE, or read it in directly as a JSON structure).
0 Comments
Answers (1)
Geoff Hayes
on 20 May 2019
James - as a first step, try using webread to read the data from the URL which you can then pass to jsondecode like
urlToFetch = 'http://fantasy.espn.com/apis/v3/games/flb/seasons/2019/segments/0/leagues/15243217?view=mMatchupScore&view=mRoster&view=mScoreboard&view=mSettings&view=mTopPerformers&view=mTeam&view=modular&view=mNav';
jsonData = webread(urlToFetch);
As for logging into the website, that may be a little more difficult. Do you know of any public APIs that would allow this? I suspect that when you login, a token of some kind would be returned that you would need to use on subsequent calls to get the other data.
3 Comments
Geoff Hayes
on 20 May 2019
Edited: Geoff Hayes
on 20 May 2019
The 401 error means that you are unauthorized and so need to provide the correct login credentials.
As for getting the content (the json) from the web browser, could you try something like
urlToFetch = 'http://fantasy.espn.com/apis/v3/games/flb/seasons/2019/segments/0/leagues/15243217?view=mMatchupScore&view=mRoster&view=mScoreboard&view=mSettings&view=mTopPerformers&view=mTeam&view=modular&view=mNav';
[stat,h] = web(urlToFetch);
jsonData = get(h, 'HtmlText') ;
using the private URL once you have logged in.
See Also
Categories
Find more on JSON Format in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!