How do I obtain data from a table from a website into MATLAB

5 views (last 30 days)
Im working on a project to track fire calls in in rockland county and to do that there is a websight that has all calls in the last 24 hours how do I collect the data from the websight into matlab. I've tred using web read but I can't find the call data in the output it gives.
Also how do I deal with repeate data everytime I webscrape the sight
  4 Comments
Star Strider
Star Strider on 25 Apr 2025
There are nly two options I can think of.
The firsi is to copy the entire table manually and paste it into a text editor or spreadsheet. This will require parsing it manually as well. Then you can read it using readtable.
The second is to contact the website owner and ask them if there is a way to access the table informaiton. There might be. Since that might not produce immediate results, unless you have an absolutely urgent need for that information, (in which instance the copy-paste-parse-readtable approach is the only option), that would be how I would approach this.
Adam Danz
Adam Danz on 4 May 2025
I got as far as extracting the table headers and then realized that the table content appears to be dynamically loaded after the page is loaded so webread will not see it. I confirmed this by viewing the page source where the table content is missing (it's visible in Inspect Element view but that's not what webread sees). You could used a 3rd party API or try to save the page in a format that could be used with MATLAB's readtable or webread.
url = "https://firewatch.44-control.net/history.html";
data = webread(url);
tree = htmlTree(data);
tableNode = findElement(tree, "table#maintable");
headerNodes = findElement(tableNode, "th");
headers = extractHTMLText(headerNodes)
headers = 6x1 string array
"Time Opened" "Time Closed" "FD" "Incident Type" "Address" "Common Name"

Sign in to comment.

Accepted Answer

Seth Furman
Seth Furman on 5 May 2025
Edited: Seth Furman on 5 May 2025
For reference, readtable now works directly on urls, but as @Adam Danz pointed out, the data on this webpage is loaded dynamically so that approach still won't work in this case.
readtable("https://firewatch.44-control.net/history.html")
ans = 0x6 empty table Time Opened Time Closed FD Incident Type Address Common Name ___________ ___________ __ _____________ _______ ___________
That said, you can manually download the rendered HTML from the webpage and read it into MATLAB using readtable.
readtable("44-Control Firewatch - Previous Incidents.txt",FileType="html")
ans = 28x6 table
Time Opened Time Closed FD Incident Type Address Common Name _____________________ _____________________ __ _________________ ____________________ ______________________________________________ "15:04:17 05/05/2025" "15:17:59 05/05/2025" 18 "Alarm Sounding" "15 FONDA DR" <missing> "14:44:17 05/05/2025" "15:34:24 05/05/2025" 6 "Automatic Alarm" "1 PHIL TISI WAY" "CLOVER STADIUM" "14:41:16 05/05/2025" "15:02:42 05/05/2025" 12 "Electric Inside" "125 ABLONDI RD" <missing> "14:11:04 05/05/2025" "14:37:58 05/05/2025" 19 "Rescue Call-Inj" "MILE RD & VIOLA RD" <missing> "14:11:04 05/05/2025" "14:37:58 05/05/2025" 20 "Rescue Call-Inj" "MILE RD & VIOLA RD" <missing> "13:40:05 05/05/2025" "13:52:38 05/05/2025" 11 "Automatic Alarm" "127 ROUTE 303" "ORANGETOWN WASTE WATER - ADMINISTRATION BLDG" "11:24:15 05/05/2025" "11:48:17 05/05/2025" 16 "Outside Fire" "244 TWEED BLVD" <missing> "09:11:49 05/05/2025" "09:34:10 05/05/2025" 22 "Automatic Alarm" "142 LAKE RD" "NYACK UFSD LIBERTY ELEMENTARY" "09:11:49 05/05/2025" "09:34:10 05/05/2025" 3 "Automatic Alarm" "142 LAKE RD" "NYACK UFSD LIBERTY ELEMENTARY" "08:31:51 05/05/2025" "08:46:17 05/05/2025" 16 "Automatic Alarm" "125 ROUTE 340" "ST. THOMAS AQUINAS, BORELLI" "08:25:41 05/05/2025" "08:54:34 05/05/2025" 4 "Structure Fire" "52 COOLIDGE ST" <missing> "08:25:41 05/05/2025" "08:54:34 05/05/2025" 23 "Structure Fire" "52 COOLIDGE ST" <missing> "08:25:41 05/05/2025" "08:54:34 05/05/2025" 44 "Structure Fire" "52 COOLIDGE ST" <missing> "01:12:26 05/05/2025" "01:32:36 05/05/2025" 19 "Automatic Alarm" "200 LAFAYETTE AV" "ROCKLAND HOME FOR THE AGED" "23:57:23 05/04/2025" "00:19:15 05/05/2025" 10 "Alarm Sounding" "221 PIERMONT AV" <missing> "21:06:54 05/04/2025" "21:37:28 05/04/2025" 12 "Electric Hazard" "55 KINSLEY GROVE" <missing>
  1 Comment
Seth Furman
Seth Furman on 5 May 2025
Looks like the raw data come from https://firewatch.44-control.net/status.json, so you can use readstruct to get the data and struct2table to convert the data into a table.
s = readstruct("https://firewatch.44-control.net/status.json")
s = struct with fields:
Timestamp: "16:22:18 05/05/2025" Interval: "30" Fire: [1x32 struct] FireOOS: [1x6 struct] Runtime: "3.66 seconds"
struct2table(s.Fire)
ans = 32x15 table
ID IncidentNumber TimeReported TimeOpened TimeClosed FD IncidentType Address Address2 Friendly NearestIntersection CallStatus Units Latitude Longitude ___________ ______________ _____________________ _____________________ _____________________ ______ _________________ _______________________ __________________________ ______________________________________________ ______________________________________________________ _______________ ____________ ________ _________ "25-093646" "2506-0341" "16:05:48 05/05/2025" "16:05:48 05/05/2025" <missing> "FD06" "Automatic Alarm" "1 PHIL TISI WAY" "N E RAMAPO, NY 10970" "CLOVER STADIUM" "STATION RD & FIREMENS MEMORIAL DR" "Under Control" {1x1 struct} 41.169 -74.039 "25-093643" "2518-0126" "15:59:28 05/05/2025" "15:59:28 05/05/2025" "16:18:16 05/05/2025" "FD18" "Rescue Call-Inj" "PIP E14 SB - OFF RAMP" "STONY POINT, NY 10980" "" "PALISADES INTERSTATE PKWY S & WILLOW GROVE RD" "Completed" {1x1 struct} 41.222 -74.035 "25-093641" "2508-0209" "15:55:39 05/05/2025" "15:55:39 05/05/2025" "16:11:56 05/05/2025" "FD08" "Alarm Sounding" "23 GLEN DR" "BARDONIA, NY 10954" "" <missing> "Completed" {1x1 struct} 41.112 -73.998 "25-093640" "2517-0373" "15:54:58 05/05/2025" "15:54:58 05/05/2025" "16:19:23 05/05/2025" "FD17" "Automatic Alarm" "46 W CHURCH ST" "SPRING VALLEY, NY 10977" "USIS 46 CHURCH STREET" "N MYRTLE AV & JOHNSON ST" "Completed" {1x1 struct} 41.113 -74.048 "25-093607" "2518-0125" "15:04:17 05/05/2025" "15:04:17 05/05/2025" "15:17:59 05/05/2025" "FD18" "Alarm Sounding" "15 FONDA DR" "STONY POINT, NY 10980" "" "ANDERSON DR & WILES DR" "Completed" {1x1 struct} 41.221 -73.993 "25-093598" "2506-0340" "14:44:17 05/05/2025" "14:44:17 05/05/2025" "15:34:24 05/05/2025" "FD06" "Automatic Alarm" "1 PHIL TISI WAY" "N E RAMAPO, NY 10970" "CLOVER STADIUM" "STATION RD & FIREMENS MEMORIAL DR" "Completed" {1x1 struct} 41.169 -74.039 "25-093595" "2512-0125" "14:41:16 05/05/2025" "14:41:16 05/05/2025" "15:02:42 05/05/2025" "FD12" "Electric Inside" "125 ABLONDI RD" "PEARL RIVER, NY 10965" "" "CARA DR & VILLA RD" "Completed" {1x1 struct} 41.072 -74 "25-093576" "2519-0116" "14:11:04 05/05/2025" "14:11:04 05/05/2025" "14:37:58 05/05/2025" "FD19" "Rescue Call-Inj" "MILE RD & VIOLA RD" "MONTEBELLO, NY" "" "VIOLA RD & CHARNWOOD DR" "Completed" {1x1 struct} 41.137 -74.111 "25-093576" "2520-0239" "14:11:04 05/05/2025" "14:11:04 05/05/2025" "14:37:58 05/05/2025" "FD20" "Rescue Call-Inj" "MILE RD & VIOLA RD" "MONTEBELLO, NY" "" "VIOLA RD & CHARNWOOD DR" "Completed" {1x1 struct} 41.137 -74.111 "25-093553" "2511-0118" "13:40:05 05/05/2025" "13:40:05 05/05/2025" "13:52:38 05/05/2025" "FD11" "Automatic Alarm" "127 ROUTE 303" "ORANGEBURG, NY 10962" "ORANGETOWN WASTE WATER - ADMINISTRATION BLDG" "JIM DEAN DR" "Completed" {1x1 struct} 41.036 -73.939 "25-093468" "2516-0063" "11:24:15 05/05/2025" "11:24:15 05/05/2025" "11:48:17 05/05/2025" "FD16" "Outside Fire" "244 TWEED BLVD" "SPARKILL, NY 10960" "" "ROUTE 9W & NIKE LN" "Completed" {1x1 struct} 41.046 -73.923 "25-093362" "2522-0062" "09:11:49 05/05/2025" "09:11:49 05/05/2025" "09:34:10 05/05/2025" "FD22" "Automatic Alarm" "142 LAKE RD" "VALLEY COTTAGE, NY 10989" "NYACK UFSD LIBERTY ELEMENTARY" "CHARLES BLVD, VALLEY RD & ROCKLAND LAKE RD, ROUTE 9W" "Completed" {1x1 struct} 41.13 -73.927 "25-093362" "2503-0077" "09:11:49 05/05/2025" "09:11:49 05/05/2025" "09:34:10 05/05/2025" "FD03" "Automatic Alarm" "142 LAKE RD" "VALLEY COTTAGE, NY 10989" "NYACK UFSD LIBERTY ELEMENTARY" "CHARLES BLVD, VALLEY RD & ROCKLAND LAKE RD, ROUTE 9W" "Completed" {1x1 struct} 41.13 -73.927 "25-093331" "2516-0062" "08:31:51 05/05/2025" "08:31:51 05/05/2025" "08:46:17 05/05/2025" "FD16" "Automatic Alarm" "125 ROUTE 340" "SPARKILL, NY 10976" "ST. THOMAS AQUINAS, BORELLI" "THORPE DR & ROUTE 303, GREENBUSH RD" "Completed" {1x1 struct} 41.042 -73.937 "25-093324" "2504-0079" "08:25:41 05/05/2025" "08:25:41 05/05/2025" "08:54:34 05/05/2025" "FD04" "Structure Fire" "52 COOLIDGE ST" "HAVERSTRAW, NY 10927" "" "COOLIDGE CT & FERRACANE PL" "Completed" {1x1 struct} 41.201 -73.98 "25-093324" "2523-0099" "08:25:41 05/05/2025" "08:25:41 05/05/2025" "08:54:34 05/05/2025" "FD23" "Structure Fire" "52 COOLIDGE ST" "HAVERSTRAW, NY 10927" "" "COOLIDGE CT & FERRACANE PL" "Completed" {1x1 struct} 41.201 -73.98

Sign in to comment.

More Answers (0)

Categories

Find more on Tables in Help Center and File Exchange

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!