Welcome to Synthiam!

The easiest way to program the most powerful robots. Use technologies by leading industry experts. ARC is a free-to-use robot programming software that makes servo automation, computer vision, autonomous navigation, and artificial intelligence easy.

Get Started
USA
Asked — Edited
Resolved Resolved by WBS00001!

Reading Through An Rss File To Get Specific Data

I have been able to get and save the attached file to my hard drive. I can read it but I can not figure out how to pull just the data I want. In this case it is a stock quote for VZ. I want to just pull out the symbol and stock price.

I would appreciate any examples of how to accomplish that.

Thank You


ARC Pro

Upgrade to ARC Pro

Become a Synthiam ARC Pro subscriber to unleash the power of easy and powerful robot programming

Load More ...
PRO
USA
#62  
And any suggestions for script improvement would be appreciated. :D
PRO
Synthiam
#63  
I see what happened - i'll post an update today for you
PRO
Synthiam
#64  
Use the latest ARC and your code will work without needing to strip any characters. I modified your EZ-Script a little...

Code:


clearvariables()

$VZFile ="C:\temp\test.txt"

FileReadClose($VZFile)

$todaydate=($year + "-" + $month + "-" + $day)
$ready=0
$thenumber=1
$sortcounter=0
$arraycounter=0
$hazard=0

definearray($aname,20,"")
definearray($amag,20,"")
definearray($adiamin,20,"")
definearray($adiamax,20,"")
definearray($ahaz,20,"")
definearray($aclosedate,20,"")
definearray($aMPH,20,"")
definearray($amiles,20,"")
definearray($abad,20,"")

$url1 = "https://api.nasa.gov/neo/rest/v1/feed?start_date="; + $todaydate + "&end_date=" + $todaydate + "&api_key=3jMUBZIcgvFXVAMouSZqlWnbjg5N1sPmgsHiEyCD"

if (FileExists($VZFile))
  FileDelete($VZFile) #Clears the file and prevents adding to it needlessly
endif

FileWrite($VZFile,HTTPGet($url1))

$endOfFile = FileReadEnd($VZFile)

repeatuntil($endOfFile)

  $line = FileReadLine($VZFile)

  $field1 = Indexof($line,"name:")
  $field2 = Indexof($line,"absolute_magnitude_h: ")
  $field3 = IndexOf($line,"feet:")
  $field4 = indexof($line,"is_potentially_hazardous_asteroid: ")
  $field5 = indexof($line,"close_approach_date:")
  $field6 = Indexof($line,"miles_per_hour:")
  $field7 = indexof($line," miss_distance:")

  if ($field1 > -1)
    $field1a = Length($line)
    $name = (substring($line,$field1+5,$field1a-7))
  endif

  if ($field2 > -1)
    $field2a = Length($line)
    $mag = (substring($line,$field2+22,$field2a-24))
  endif

  if ($field3 > -1)
    $line = FileReadLine($VZFile)
    $field3a = IndexOf($line,"estimated_diameter_min: ")
    $diamin=(substring($line,$field3a+24,6))

    $line = FileReadLine($VZFile)
    $field3b = IndexOf($line,"estimated_diameter_max: ")
    $diamax=(substring($line,$field3b+24,6))

  endif

  if ($field4 > -1)
    $haz = substring($line,$field4+35,3)
  endif

  if ($field5 > -1)
    $field5a = indexof($line," ,")
    $closedate = substring($line,$field5+20,$field5a-21)

  endif

  if ($field6 > -1)

    $MPH = substring($line,$field6+15,7)

  endif

  if ($field7 > -1)
    $line = FileReadLine($VZFile)
    $line = FileReadLine($VZFile)
    $line = FileReadLine($VZFile)
    $line = FileReadLine($VZFile)
    $field7a = Length($line)
    $miles = (substring($line,7,$field7a-8))
    $ready = 1
    $field7 = -1

    # print($name)
    # print($mag)
    # print($diamin)
    # print($diamax)
    # print($haz)
    # print($Closedate)
    # print($MPH)
    # print($miles)
    # sleep(3000)
  endif

  if ($ready = 1 )

    $arraycounter=$arraycounter+1
    $aname[$arraycounter]=$name
    $amag[$arraycounter]=$mag
    $adiamin[$arraycounter]=$diamin
    $adiamax[$arraycounter]=$diamax
    $ahaz[$arraycounter]=$haz
    if ($ahaz[$arraycounter] = "fal")
      $hazard = $hazard
    ELSE
      $hazard = $hazard + 1
      $abad[$arraycounter]=$name
    endif
    $aclosedate[$arraycounter]=$closedate
    $aMPH[$arraycounter]=$MPH
    $amiles[$arraycounter]=$miles

    $ready=0

  endif

  $endOfFile = FileReadEnd($VZFile)

endrepeatuntil


filereadclose($VZFile)

repeatuntil($sortcounter = $arraycounter)

  if ($amiles[$thenumber]< $amiles[$sortcounter+1])
    $thenumber=$thenumber

  ELSE
    $thenumber=$sortcounter+1
  endif
  $sortcounter=$sortcounter+1
endrepeatuntil


if ($hazard = 0)
  $hazardstatement =(" None of these encounters are rated as potentially hazardous.")
ELSE
  $hazardstatement = ("with " + $hazard + " rated as potentially hazardou.")
endif

saywait(" There are " + $arraycounter + " Near Earth Asteroid Fly buys " + $hazardstatement)

if ($arrayCounter > 0)
  saywait(" The asteroid designated" + $aname[$thenumber] + " will safely pass by earth at a distance of " + round($amiles[$thenumber]) + " Miles.")
  saywait("It is estimated to be between " + round($adiamin[$thenumber]) + " feet and " + round($adiamax[$thenumber])+ " feet in diameter and, is traveling At a speed of " + (round($aMPH[$thenumber])) + " MPH.")
endif
PRO
USA
#65  
DJ. Did you actually get the script to work? I downloaded and installed the latest version of ARC. Used your script and it downloads and writes the file, but cant find any info for today and I know there is. Am I doing something wrong?
PRO
USA
#66  
Ok, I see. I need to reparse through the data to find my values. Much work. I have probably 5 others I need to redo. I did get this one working though. Thanks.
#67  
I'm a bit late to this party. Yea! the left paren issue has been resolved! Unfortunately there are other things commonly found in the strings which make up a web page which will cause problems and crashes so filtering will still be needed in other cases. For example quote marks, or LF and CR in cases in which the user wishes to take in the entire web page contents at once into a single variable.

If you look at the output file you will see that the search terms like name: and Absolute_magnitude_h: are now (without filtering) name : and absolute_magnitude_h : with a space before the colon which the filter program must have previously taken out. So you would need to change those search words, as well as the others, to include the space for it to work properly. That's why they are not being found currently.

EDIT: Ah, I see dbeard, you have found the problem while I was checking things out and creating this reply.
PRO
USA
#68  
WBS00001, are there other things your program was clearing out that is going to cause a problem? With the update your Scrub program no longer works.

I have got the script working after fixing a lot of the parsing.:)
#69  
@dbeard
In what way is the Scrub program not working?

The Scrub program does some pre-emptive character replacement in addition to the left paren removal. It also deletes double quotes and changes E to e. It gets rid of quotes because strings in the Scripts are bounded by them. So any in a string read from a web page will cause a crash or truncated string. This isn't a bug though, just a fact of life. Lower casing E prevents the problem when an uppercase E is followed directly by a number. The script interpreter will try to treat it as an E-notation value and crash in so doing. Also a backslash can sometimes cause problems so they are removed, as well as the semi-colon.

As your project currently stands it seems to be free from any of those problems, given the web page you are using.

I have a new version of the scrub program which is more versatile than what you are currently using should you be interested. It is used pretty much like you are using what you have now. Switching to it would not be too difficult. If you post your project to the cloud I could download it and modify it to use the new stuff, then upload it back.
PRO
Synthiam
#70  
Thanks WBS, the capital E has been resolved for the next release
PRO
USA
#71  
WBS00001 - I will let you know once in the cloud.
PRO
Synthiam
#73  
String quotes are working as well for next release. Need a few more tests before i put it online

User-inserted image
#74  
@dbeard
After giving it more thought, and the fact that DJ will soon be fixing the quote problem and the capital E problem, I'm thinking you will be better off without the Scrub program at all. Just do things like DJ last showed and let it go at that. Your only real problem was with the left paren anyway. At least with the script you posted. I see you have other scripts for other web pages as well. I haven't checked them out.

The only other thing the Scrub program can do for you is to limit the amount of text gotten from the web page, That can be helpful in that you can avoid having to wade through many lines of text to get to the text with the information you want. That can speed up response time. If there was an instruction to set the file pointer to a specific position in the file, that too could, effectively, be done in the script.

Of course, it can still provide substitutions, exchanging a character or group of characters to another character or group of characters. That includes non-printable characters. But I think now most, if not all, substitutions can be made in the script instead. Though there is still the problem of LF or CR characters crashing the script if loaded into a variable. That would happen if you decided to load the entire web page at once into a single variable using the FileReadAll script instruction. Reading it line by line avoids that, however.
PRO
Synthiam
#75  
Is that a feature request for the httpget command? Is limit the amount of data?
PRO
USA
#76  
WBS0001 - Ok, thanks for all the help. DJ, can you make the data limit something the user can decide, like an added parameter to the command. Sometimes good to get all the data and sometimes not. Also, DJ, could you delete my app from the cloud. please.
#77  
@DJSures
If it were a feature, it would be good if there were 2 parameters associated with the HTTPGet function. Right now the Scrub program can limit the data gathered from the web page in 2 ways. One is by supplying an integer specifying a start position from which to begin and another integer specifying how much to get starting at that point. If the amount to get is beyond the end of the web page text, it simply gets whatever text there is instead.

Alternatively, two strings can be specified. The Scrub program uses the first string as a search parameter to locate the start position that way, and the other to specify an end position to indicate where to end. Additionally, the search for the end position does not begin until after the point at which the start string was found in the page text (plus the length of the start string). However, if the start position string is not found, then the end string is ignored and the entire web page is returned instead. If the start string is found, but not the end string, then the web page text is gathered to the end of the web page. It also pops up error messages should either of those conditions occur. There is a way to suppress those error messages should the user desire not to have them appear.

The program allows for mixing the two methods as well. For example a string to find the start position and an integer to specify how much text to get from the point at which the start string was found.

So, basically, to do the same thing, the two parameters would have to be such that they can take either an integer or a string.
#78  
@dbeard
You can delete the project yourself by going to it in the cloud. A delete option should appear for you since you posted it.
PRO
Synthiam
#79  
Thanks! I don't think the parameters are necessary after reading your description. The EZ-Script SubString will do the same thing

Code:



# Get 50 characters after the 10th character
$x = SubString(httpGet("http://www.google.ca";), 10, 50))


*Note: that wont' work for you currently until the next ARC is updated because of the quote issue with current ARC. But it works on mine:D
#80  
@DJSures
Since we are on the topic of fixing errors in the script, I would like to take this opportunity to reiterate the fact that the script will also crash if the strings from a web page contain Line Feed or Carriage Return characters and the entire web page is loaded at once using FileReadAll. Correcting this would be helpful to folks who would like to work with the entire text from a web page at once from a single variable, as opposed to, reading the file line by line.

EDIT: Removed the request for optional parameters in FileRealAll since the SubString function will do that as well.

Thanks, once again, for all your efforts.
PRO
Synthiam
#81  
You bet! Line feed has also been resolved
Login to post a comment