Blog articles

How to collect more than 1 million survey responses per day

Posted on 5/3/2011 by Jim in category: survey software articles
What survey software should you choose to collect 1,400,000 (1.4 million) survey responses per day? Novi Survey of course. Collecting this much data requires special care both in the setup of the online survey software and in choosing and in configuring the hardware running the various components of Novi Survey.

First, the software should be turned into high load mode in the configuration page. The high load mode is available in the enterprise release of the Novi Survey tool. The high load mode enables several levels of caching that are critical to reducing the number of searches in the database, which can degrade the performance under loads of tens of thousands of responses collected each hour. Once the high load mode is turned on, changes to surveys are no longer reflected immediately to participants. This lag happens because much of the survey definition data is cached by the application and changes are not reflected immediately in the data caches. To make sure that all recent updates to a survey are shown to participants, you should go to the survey list screen and preview the survey. Using the “preview” tool has the side effect of clearing all cached data for that survey. Therefore, participants accessing the survey after it was previewed will see its most up to date definition.

On the hardware side, the survey software should be set up to run in a web farm. The database tier should be located on a powerful server with fast disks to minimize data read and write time. As for any database setup, the storage for the server should be set with some level of redundancy to be able to cope with single or multiple disk failures. A simple RAID 5 configuration will be appropriate in most situations. There should be multiple application or web servers running the web and business logic components. The number of servers required depends primarily on the computational power (i.e., CPU and overall computation speed) available on each server. It is possible to mix and match servers with different levels of capabilities. However, running in such an heterogeneous environment requires that the load balancing component of the whole setup be capable of dispatching the workload based on the capacity of each server. In essence, through load balancing, the servers with greater capability should handle a greater fraction of the overall work load.

There are additional details required to make this kind of setup work most efficiently, which depends on the type of surveys you intend to run with our online survey tool. You should contact our support prior to launching the surveys so that we can advise you on the optimum configuration for your application.

As always, happy surveying.

Assessment Software | Survey Software Blog | Survey Sitemap | Privacy Policy
Copyright © 2007 - 2024 All rights reserved.
3rd Millennium, Inc. - 125 Cambridge Park Drive, Suite 301 - Cambridge, MA 02140, USA. Phone: 781-547-7739