Welcome to Synthiam!

The easiest way to program the most powerful robots. Use technologies by leading industry experts. ARC is a free-to-use robot programming software that makes servo automation, computer vision, autonomous navigation, and artificial intelligence easy.

Get Started
USA
Asked
Resolved Resolved by ptp!

Filereadline Getting Error Data Is In Wrong Format

I have created a file by downloading data from the web.

this is the line of code

FileWrite($DataFile,HTTPGet($url1))

Where $DataFile is the file location on my harddrive and url1 is the where the data is on the web.

The data gets successfully wrote to the hard drive. 

Now my next line of code is $line = FileReadLine($DataFile). 

The first line of the Data file is:  <!DOCTYPE html>

This fails with the error Input string was not in a correct format.  

Now this used to work but with ARC this fails.

What format should the Datafile be in for this to continue?

Related Hardware EZ-B v4

ARC Pro

Upgrade to ARC Pro

Unleash your creativity with the power of easy robot programming using Synthiam ARC Pro

Load More ...
PRO
USA
#15  
[quote][/quote]I am able to read and write the data file.  But now I need to search through it and find a string.

I am using
               var linein = File.readLine(dataFile);

                var n = dow.search(linein);  (this is line 20 the error is referencing.)

but receive the following error:

Execution Error Line 20 Col 2 - Line 1: Invalid regular expression

another bit of help?
PRO
USA
#16   — Edited
a working example:

Code:

var baseUrl = "https://synthiam.com";
var url1 = "https://synthiam.com/Community/Questions";

print("Downloading questions");
var content = Net.hTTPGet(url1);

print("Content length=" + content.length);

print("Questions found:");

var pos = 0;
do
{
  //This method returns -1 if the value to search for never occurs.
  pos = content.indexOf("/Community/Questions/", pos);
  
  if (pos>=0)
  {
      beginPos = pos

      //search for a double quote after finding the begin string
      pos = content.indexOf("\"", beginPos);
      if (pos>0)
      {
        endPos = pos;
        var link = content.substring(beginPos, endPos);
        print (baseUrl + link);
      }
  }
}
while(pos>=0);
the above code downloads Synthiam's front page questions and parses que questions links using a indexOf.

The method search uses regular expressions.

help with JS:
https://www.w3schools.com/jsref/jsref_indexof.asp


@DJ:
I believe there are some bugs with File class exported to Jint.

*** code edited ***
PRO
USA
#18  
This is the html content:
User-inserted image


Red is the "begin text" and Blue is the end
PRO
USA
#19  
special attention to substring method:
https://www.w3schools.com/jsref/jsref_substring.asp

Definition and Usage
The substring() method extracts the characters from a string, between two specified indices, and returns the new sub string.

This method extracts the characters in a string between "start" and "end", not including "end" itself.
PRO
Synthiam
#20  
What bugs are you noticing with the File class? And what version of ARC are you using? (latest?)
PRO
USA
#21  
weird stuff going on, maybe is a consequence of a single bug.
I'll post simple code to  demo the issue.
PRO
USA
#22  

Code:

function PrintLines(filename) 
{
  var lineIx = 0;
  while (!File.isReadEnd(filename)) 
  {
    var line = File.readLine(filename);
    print(lineIx + " line:" + line);
  }
}


var filename="c:\\temp\\lines.txt";

var now = new Date().toLocaleString();
if (!File.exists(filename))
{
  print("file does not exist");
  File.appendStringLine(filename, now + ">>" + "First time");
}
else
{
  print("file exists");
  File.appendStringLine(filename, now + ">>" + "foo");
}

print("reset read file");
File.readReset(filename);

print("PrintLines");
PrintLines(filename);

print("add bar");
File.appendStringLine(filename, now + ">>" + "bar");

print("PrintLines");
PrintLines(filename);

first time (file does not exist)
User-inserted image


Close ARC, Open ARC run the script:
User-inserted image


Q: without a close and explicit open how you control reads and writes ?

Code:

File.appendStringLine
Locks the file, and you need to close ARC.
PRO
USA
#23  
Reading from a File locks the file too so you need to close ARC.
User-inserted image
PRO
USA
#24  
PTP

In your answer in #16.  I think I follow but what I cant figure out is why doesn't your code download all the questions.  Why does it stop after 9.

I don't see anything in your code that says just show the first 9 questions.
PRO
USA
#25   — Edited
That's a different question, the website content is dynamic, does not make sense to list all the questions when a page is loaded. 
Look to twitter when you scroll down to the last visible post, the page requests more content from the server same thing here.

synthiam page captures the document page scroll:

Code:

$(document).ready(function() {
    if ($("#sidebarsearchsort").change(function() {
        endOfResults = !1;
        replaceSearchContent()
    }),
    $("#search-result").length)
        $(document).on("scroll ready", function() {
            element_in_scroll("#search-result .content-card:last-child") && (loading || addSearchContent())
        })
});
and calls the function addSearchContent

Code:

function addSearchContent() {
    searchContent(function(n) {
        $("#search-result").append(n)
    })
}


function searchContent(n) {
    if (endOfResults !== !0 && loading !== !0) {
        var t = typeof compact == "undefined" ? !1 : compact
          , i = typeof subCat == "undefined" ? null : subCat
          , r = typeof noLabel == "undefined" ? !1 : noLabel
          , u = typeof sort == "undefined" ? $("#sidebarsearchsort").val() : sort
          , f = typeof exSubCat == "undefined" ? null : exSubCat
          , e = typeof searchQ == "undefined" ? null : searchQ;
        loading = !0;
        addLoadingCards(t);
        $.ajax({
            method: "POST",
            data: {
                __RequestVerificationToken: $("[name='__RequestVerificationToken']").val(),
                pageNumber: pageNumber,
                sort: u,
                type: cat,
                referenceId: $("#selected-behavior-control option:selected").val(),
                compactView: t,
                subCat: i,
                noLabel: r,
                excludeSubCat: f,
                searchTerm: e
            },
            url: "/api/search/byType"
        }).done(function(t) {
            if (loading = !1,
            removeLoadingCards(),
            t === "") {
                endOfResults = !0;
                return
            }
            n(t);
            pageNumber = pageNumber + 1;
            var i = (JSON.stringify(t).match(/class=\\"content-card |class=\\"content-card\\"/g) || []).length;
            i < 9 && (endOfResults = !0)
        })
    }
}
the function executes an ajax call to the server /api/search/byType and the server returns more html content (i.e. questions) that content will be appended to a specific document placeholder:

Code:

$("#search-result").append(n)

short story when you request the questions page a fixed number of questions are returned, after each scroll  more questions are retrieved from the server.

you can read more about "infinite scroll":
http://harshw.com/infinite-scroll-using-jquery-asp-net-mvc-mustache-entity-framework-paginate/
PRO
USA
#26  
You lost me at infinite scroll.

But I do understand this (I think) the code only reads a page at a time.  It will take me some time to digest the above.

one last question for the day.  BTW.  I have it working now and getting the data from the page thanks to your help.

Here is the question.    this is what the data looks like:    >5:26 PM</td>   The data I want is the 5:26 PM.  But depending on the time of day it could be 10:26 PM.  Is there a one-liner function that can pull the 5:26 PM or 10:26 PM out from between those markups really easy without including the other stuff?
PRO
Synthiam
#27   — Edited
If the file is locked, you have to close the file from reading with the close command. Because reading the file keeps the file open while you read it. 

Code:


File.readClose( filename );
PRO
USA
#28  
I missed that method, but i don't see an equivalent writeClose
PRO
Synthiam
#29  
Because there is no write. It's append. Appending just keeps appending to an existing file or creates a new one. There's no reason to keep a file open for appending. The only point to keeping the file open for reading is to know the position.

I've added two new methods for getting the current read position and getting the file length. It'll be in today's release.
PRO
USA
#30  

Code:

File.appendStringLine
This commands locks the file and does not release the file. Is that an expected behavior ?
PRO
USA
#31   — Edited
@dbeard:

User-inserted image


Example to capture the red part 

The forum break's the JavaScript code, posted code picture:

User-inserted image

 
@DJ 
Please read my previous post!
PRO
USA
#32  
Here is my program for getting dow jones data.  I know it is a mess, but it works.

Code:

var url1 = "http://www.marketwatch.com/tools/marketsummary/indices/indices.asp?indexid=1&groupid=37";

print("Downloading Data");
var content = Net.hTTPGet(url1);

print("Content length=" + content.length);

print("Dow Jones Information Found:");

pos = 0;
do
{
  //This method returns -1 if the value to search for never occurs.
  var pos = content.indexOf("/investing/index/djia", pos);
  
  if (pos>=0)
  {
      beginPos = pos

      //search for a double quote after finding the begin string
      pos = content.indexOf("\"", beginPos);
      if (pos>0)
      {
        endPos = pos;
        var link = content.substring(beginPos, endPos);
        var volume = content.substring(endPos+144, endPos+153);
        var tradedate = content.substring(endPos+193, endPos+205);
        var tradetime = content.substring(endPos+245, endPos+255);
        var amount = content.substring(endPos+297, endPos+309);
        var tradepercent = content.substring(endPos+352, endPos+361);
                
        place = 0
        var startplace = volume.indexOf(">",place);
        var endplace = volume.indexOf("<", place);
        var fvolume = volume.substr(startplace+1, endplace-1);
        
        place = 0
        var startplace = tradedate.indexOf(">",place);
        var endplace = tradedate.indexOf("<", place);
        var ftdate = tradedate.substr(startplace+1, endplace-1);
        
        place = 0
        var startplace = tradetime.indexOf(">",place);
        var endplace = tradetime.indexOf("<", place);
        var fttime = tradetime.substr(startplace+1, endplace-1);
        
        place = 0
        var startplace = amount.indexOf(">",place);
        var endplace = amount.indexOf("<", place);
        var famount = amount.substr(startplace+1, endplace-1);
        
        place = 0
        var startplace = tradepercent.indexOf(">",place);
        var endplace = tradepercent.indexOf("<", place);
        var ftpercent = tradepercent.substr(startplace+1, endplace-1);
        
        print("Date:    " +ftdate);
        print("Time:    " +fttime);
        print("Volume:  "+fvolume);
        print("Amount:  "+famount);
        print("Percent: "+ftpercent);
        
        halt();
      }
  }
}
while(pos>=0);
PRO
USA
#34  
Thanks everyone for the help.
Login to post a comment