Most site search providers face the handicap of not being able to crawl JavaScript websites. Of course, searching can still be done within JavaScript pages, but sending data must be done via the API – a more time-consuming process than having the crawlers do the work automatically. Fortunately, Cludo has again led market innovation and come up with a simple solution to this problem.
Introducing the latest update to our service: asynchronous crawling!
Why is this important?
Understanding the value of this feature requires a little background on the history of webpages. Back in the day, when websites were developed they were comprised of simple and mostly static HTML pages used only to display information. Navigating through different links on the site required that the site reloads each time you entered a new page. Interaction with sites like these was not so much interactive but instead served as passive mediums for presenting information.
As an answer to the stiff, static pages of the past, dynamic webpages using JavaScript were introduced. These webpages are programmed to load only the data needed when links are clicked. They also provide a more detailed and interactive experience for the users where it can respond to and update information based on users’ navigation in real-time. Think about sites like Facebook where you can move from page-to-page without having the top navigation bar load each time. Or consider the simplicity of ‘liking’ or commenting on a post and having those responses display immediately – no page reload. This interactivity is becoming increasingly necessary for sites looking to provide an enhanced and fluid user experience.
Asynchronous crawling = improved search experience
It cannot be stressed enough how important user experience is to the functionality and efficacy of your site. People are increasingly inclined to using sites that respond in real-time and provide paths of least resistance for tasks like form completion or forum interaction. Websites created through JavaScript are becoming more prevalent in web design, so it only makes sense to include a similarly compatible and responsive onsite search.
Dynamic content should still be easily searchable and the implementation of that search should be just as easy. Our new asynchronous crawling will be able to support those JavaScript pages and provide your dynamic sites with an effortless search experience optimized for your site.
Learn more about asynchronous crawling by reaching out to our Sales team via email at support@cludo.com.