Website Tech

Category: Small Business

  • iMacros for Retrieving Information from Multiple Web Pages

    iMacros is a great tool for grabbing information from web sites. People often use it for grabbing the latest stock prices, or getting the latest prices on many products. Putting the data into a spreadsheet-compatible file is easy.

    You can also use iMacros to fill out forms from a spreadsheet (.csv) text file.

    iMacros is a free plugin for Firefox (or IE), or there is a more powerful paid version.

    You specify the information to grab using a simple scripting language, that can search a web page for HTML tags or attributes. So, you could search for the the paragraph with an ID, and grab the text. Or look for a link that starts with a particular domain, and grab either the link or the text.

    iMacros automatically handles looping through all the occurrences on a page (for example, each row of a table), the most common need.

    The Firefox version lets you control switching tabs, to grab the data from one tab and write it to a form on another tab. (The form could process the data and save it to a database, or anything else you could program a form processing routine to do.)

    But it doesn’t have a full programming language, with testing variables and calling subroutines based on the test. It doesn’t have FOR loops, or WHILE loops. So, you can grab all the prices on one page, or keep grabbing one value from pages as long as there is a “Next” link. But you can’t easily do both. You can grab and save All the values, but not save only the ones that meet your conditions.

    Well, iMacros for Firefox has a Javascript interface. With that, you can have Javascript control the looping, testing for special conditions, massaging the data, checking for errors. You can pass values from Javascript to iMacros, and return extracted information to Javascript.

    I’m going to show some code snippets for using loops to pass values to iMacros scripts and back to Javascript, and testing for errors.

    So you have some idea what the script does, in my task, one web page lists the MapID of each Map in a section of the County. Another web page lists the Parcels contained in a specific Map. Yet another web page shows what I want, details about a Parcel. I then have iMacros put the details in a custom form, and submit the form data to a PHP program that writes it to a MySQL database.

    Since each section of the county could have an unknown number of Map IDs, and each Map would have an unknown number of Parcels, and each Parcel has information on several web pages, and since there is so much information to collect it couldn’t be done in one day (probably non-stop gathering for 4 months!) and therefore I needed to have the scripts keep track of what had been completed, it was too complex for any iMacros script. But with the Javascript interface controlling 6 scripts, it works!

    mapidnum = 1
    /* Javascript uses labeled loops so you can break out of multiple loops */
    loopm:
    do {
    iimSet("mapidnum", mapidnum)
    /* get the nth map id */
    iret = iimPlay("macro-that-finds-each-map-id.iim")
    if (iret == -101) {
    /* user pressed Stop button */
    break loopm;
    }
    mapid = iimGetLastExtract() /* 849/849010100 */
    if (mapid == "#EANF#") {
    continue;
    }
    iimDisplay("TownshipRange "+t+"("+l+") SectionCount:"+s+" Section "+sections[s]+" Township "+tr[0]+" Range "+tr[1]+" MapIDnum: "+mapidnum+" MapID: "+mapid+" TotAPN="+totalapnsfound);
    mapid = mapid.slice(4,13) /* 849010100 */
    iimSet("mapid", mapid)
    /* read number of parcel IDs for this map ID */
    iret = iimPlay("macro-that-finds-how-many-detail-records-for-map-id.iim")
    apnsfoundstr = iimGetLastExtract()
    if (iret == -101) {
    /* user pressed Stop button */
    break loopm;
    }
    n = apnsfoundstr.indexOf(" APN(s) Found",0)
    if (n != -1) {
    apnsfound = parseInt(apnsfoundstr.slice(0,n))
    } else apnsfound = 0
    loopn:
    for (n=1; n<= apnsfound; n++) { iimSet("parcelloop", n) iimDisplay("TownshipRange "+t+"("+l+") SectionCount:"+s+" Section "+sections[s]+" Township "+tr[0]+" Range "+tr[1]+" MapIDnum: "+mapidnum+" MapID: "+mapid+" TotAPN="+totalapnsfound+" APN "+n+"("+apnsfound+")") iret = iimPlay("macro-that-reads-detail-record.iim") if (iret == -101) { /* user pressed Stop button, break out of labeled loop (both FOR loop and DO...WHILE) */ break loopm; } if (iret == -933) { /* when disconnect WiFi get: -933 Error Loading Page */ /* do something to retry, or run a script PROMPT "Error Loading Page (WiFi Error?)" */ continue; } parceldash = iimGetLastExtract(1) /* have single parcel number (with dashes, maybe with split letter), next step needs parcel number without dashes */ parcel = parceldash.replace(/-/g,''); /* take all values so far, pass to iMacros to put into Form for writing to Database */ iimSet("county", county) iimSet("parcel", parcel) iimSet("parceldash", parceldash) iimSet("mapid", mapid) iimSet("township", tr[0]) iimSet("range", tr[1]) iimSet("section", sections[s]) iimDisplay("TownshipRange "+t+"("+l+") SectionCount:"+s+" Section "+sections[s]+" Township "+tr[0]+" Range "+tr[1]+" MapIDnum: "+mapidnum+" MapID: "+mapid+" TotAPN="+totalapnsfound+" APN "+n+"("+apnsfound+") Parcel:"+parcel); iret = iimPlay("macro-that-reads-parcel-data-puts-it-in-form-that-writes-to-MySQL.iim") } /* end for APNs found */ mapidnum++ } while (mapid != "#EANF#")

    iMacros web site has full documentation on all the commands, including iimSet (sets a variable for use in an iMacros script), iimPlay (play an iMacros script), iimGetLastExtract (bring extracted data into Javascript), and iimDisplay (display text in a status area of the screen).

    In iMacros scripts you would access the variables (set with iimSet).

    ' macro-that-finds-each-map-id.iim uses the parameter like this:
    ' Find the nth Map ID. The page displays a link to a PDF of the map; the Map ID is the text of the link.
    TAG POS=R{{mapidnum}} TYPE=A ATTR=HREF:http://156.42.37.20* EXTRACT=TXT

    ' macro-that-reads-detail-record.iim uses the parameter like this:
    ' Find the nth Parcel Number, displayed as a link to the first page of the parcel's information (the County Assessor's page has links to the County Treasurer's info for this parcel).
    TAG POS=R{{parcelloop}} TYPE=A ATTR=HREF:/Assessor/ParcelApplication/Detail.aspx* EXTRACT=TXT

    Want Me to Help?

    If this is giving you ideas for grabbing information you could use, I can probably write iMacros scripts to get what you need, into a database or web page. Contact Me so we can talk.

  • How does SPIRIT make groups powerful?

    Success

    Partnership

    Imagination

    Respect

    Integrity

    Trust

  • Implement Your Marketing and Training Programs

    A lot of people would love to market more effectively. Or take powerful actions from that real estate workshop. Or take their team to the next level, using the new team management course you took.

    If you are thinking about Internet marketing in some way, or are actively marketing, I would like to talk with you.

    Note: If you have the new Product Launch Formula version 3, I’m very interested in helping with your launch. I am currently (July 2010) managing all the tasks for a launch, and as I do this launch I can begin helping you get ready for yours.

    Implementing What You Learned

    Very few people who take a big workshop, or buy a powerful training program, or get a home-study course, whether it’s on Internet marketing, or real estate, or sales, or management, ever get most of the great material implemented. “Great course! Uhh, what do I do now?”

    I’m very good at making clear action plans from “all those courses we’ve taken”, and leading people through them, step by step, even through the scary parts, so we can get what we wanted when we bought the courses.

    Making the Computer Stuff Much Easier

    I’m also good with the “computer stuff” that’s essential for marketing well (installing scripts, setting up blogs, starting web sites, getting information into and out of databases, editing audio). The stuff that so many people dread, or that stop people even when they know what to do, because it is so much work.

    I am an expert at using WordPress for business, for marketing, in ways that are easily used by everyone involved.

    I also have Butterfly Marketing software, that many top marketers use to implement their marketing and product launches.

    Most people are not interested in doing that computer stuff. They’ll struggle through it if they have to. And many people who could do it, shouldn’t be putting a lot of time into doing it, when it would be better for them to focus on what is next to build their business.

    Let’s talk!

    As we talk, we will quickly find out how I can help you get what you want from that big program you took, and what implementing it well will bring your organization.

    George

    Contact Me

  • Small Business / Entrepreneurship Products

    Enjoy these Small Business and Entrepreneurship products, you can download and start using them now, to improve your business.

    Please add comments about the ones you get and use. What do you like (or dislike) about them?

  • Make decisions easy for customers

    How much is what I want???
    What do I get with that???

    Make it easy for customers to do business with you.

    Cell phone companies are (usually poor) examples of how to make shopping easy.

    Instead of showing the price for what I want, the web site says price “requires one of these plans”. Well, I don’t know what that plan is, and I don’t know how much that plan costs. Then I go look at what that plan is, and it gives options — I still don’t know what my plan is going to cost.

    Here’s what I want to know:
    – How much is it?
    – How much is the difference in price from my current plan?
    – What is the difference in features from my current plan?

    Having the web site clear is even more important when I call customer service. Many companies have customer service read the web site, of course. If the web site isn’t clear, phone conversations will be even less clear than looking at the web site, since I’d have to (try to) remember the numbers they quickly read to me.

    What do you want? (I select it)

    Here’s what you’ve selected and how much it costs, and here’s how your plan will be different than what you have now.

    Here’s some options and how much your plan would cost with each one

    From what you’ve selected, (repeat previous step)

    Place order

    I want to make 600 to 800 minutes of calls per month, send 200-500 text messages a month, and occasionally be able to check a web site. Got that? Ok, now how much does that plan cost? Oh, that’s too much, how much with this smaller option?

    I want answers in this format:
    Great, if you want X minutes phone calls, unlimited text messaging and 5GB web access, here’s the price for the plan you’ve selected.
    (If you talk to customer service, say you want the “ABC Plan with DEF”).
    This is $X more than your current plan.
    From this selection, here are some other options to consider:
    a: unlimited this $/month (show total and difference)
    b: unlimited that
    c: more limited this
    d: more limited that

    If the customer says something is important to them, mention it in every plan description. If being able to download a song is important to me, don’t neglect mentioning that, just because I select a plan that includes it. “Download a Song (included with unlimited web access)”.

    Trip planning is important to me; is that included with the web access? Is it separate? How much does it cost without web access, how much with web access? How about “You said trip planning was a feature you are interested in. This plan includes it; or, this plan doesn’t include it, $X to add it”.

    Simple. Clear. Easy.

    Don’t clutter up “here’s what you get” with every feature under the sun. Most of the features are unimportant to customers. The place for listing every feature is in the early “what do you want” phase, not the “are you ready to order” phase.

    “Oh, I really want voice mail, does this plan have voice mail?” Well, all our plans have the same voice mail. “Fine, I’m checking I want voice mail, so show it”. Don’t whine to me about “it’s in the All Plans Offer section”. “It’s important to me, show me what’s important to me, and don’t show me this other junk”.

    “OK, we’ll show the list of all features available, and you can check off what’s important to you, and we’ll show you each time whether the plan you’ve selected has that feature or not. We’ll even organize the features in sections, so your screen isn’t cluttered.”

    Make it easy for customers to do business with you.