Last updated on December 17, 2013

EMBs must ensure that there is appropriate and systematic testing of electronic voting and counting systems in the period before an election so that problems can be highlighted and addressed in a timely fashion before Election Day. EMBs should also provide access to independent experts to review the source code in order to engender transparency and build confidence in the electronic systems. EMBs may also require independent bodies to certify the electronic voting and counting systems prior to their use. Both testing and certification are time-consuming processes, and EMBs should ensure sufficient time before Election Day for these steps in the process to take place. Observers and parties should ensure they have the expertise and capacity to comprehensively inspect the source code and assess the testing and certification processes.

Testing and Source Code Review

Ensuring that electronic voting or counting systems function correctly and generate accurate results based on the votes cast is critically important. Not only must election management bodies ensure this, but they also must convince key electoral stakeholders that this is the case so that they will trust and accept the results. Unlike other electronic transactions, one cannot check afterward that his or her vote was recorded correctly. For example, with electronic banking, people can check their statements to see if any incorrect transactions were made and can have mistakes corrected. The need for a secret vote denies the possibility for this level of transparency.

As a result, the EMB needs to make additional efforts to test the electronic voting or counting system before it is used to ensure that it works correctly.³⁷ Figure 5 in the Overview section shows the different kinds of tests that the Council of Europe recommends for electronic voting and counting systems; these include acceptance testing, performance testing, stress testing, security testing, usability testing and source code review.

All of these tests will be conducted by, or on behalf of, the EMB. The more these tests can be conducted by the EMB, the better, as long as it has the competency to do so. If any aspect of testing is outsourced, EMB personnel must remain engaged and provide oversight of the testing process. From a transparency and confidence-building perspective, it is also useful to have an external, independent body conduct some level of testing. In the US, local EMBs carry out testing before each election. In Maryland, this testing consists of preparing and configuring the machines, casting hundreds of votes on each voting machine, and producing and checking results on the voting machine as well as through the central tabulation system, before clearing the voting machines of voting data, sealing them and securing them so they are ready for use in the election. 

While different EMBs will take varying approaches to the testing regime that is used, it is vital that the EMB does some level of testing and that testing is done before each election. Testing before each election is necessary to check the election-specific configuration and also to deal with any technology changes, which is especially important for Internet voting where new browsers as well as updated versions of existing browsers may need to be accommodated.

Full system testing also needs to take place sufficiently in advance of elections to enable the remedy of any problems encountered. It is also prudent to do a final check of equipment closer to Election Day. In the 2010 Philippine election, during which electronic counting machines were being used for the first time, the machines were scheduled for final testing and sealing one day before the election. The COMELEC IT Department decided to test some machines earlier and discovered less than a week before the election that a bug in the configuration of the scanning software would cause the machines to register votes incorrectly. The decision to do early testing and sealing detected the problem in time, so that new compact flash cards could be distributed nationwide, rescuing the election from disaster.

Access to the source code for electronic voting and counting applications may also be made available so that independent experts can check that no errors exist in the source code (see the previous discussion of open source code in the “Security Mechanisms” section). Additional scrutiny of the source code may help to identify the existence of any errors, oversights or malicious code, but will also importantly help to build confidence in the electronic voting or counting systems by increasing transparency. 

Fully open source code is not necessary to provide these confidence-building mechanisms, but it is the more preferable option. Should open source code not be used, experts representing key electoral stakeholders (political actors and civil society) should be allowed sufficient access to review the source code and should not be restricted in reporting their analysis of its content by the use of any nondisclosure agreements. The EMB may also decide to engage an external body to conduct an independent review of the source code as a confidence-building measure.

All of the reports on the testing of electronic voting or counting systems should be made available for review by political actors and observers. Again, this transparency will help to build confidence in the system.

It is important to recognize that conducting these different kinds of tests takes a significant amount of time and resources. Electronic voting and counting systems are complex; and especially when new systems are developed, they will often contain errors that need to be corrected. Each time an error is identified and corrected, it may be necessary to conduct the full test process again, as even a small change may lead to unforeseen consequences. Therefore, sufficient time and resources must be allocated for this testing to take place, as well as for any corrections and retesting to be implemented.

Certification

In addition to comprehensive testing of electronic voting and counting technologies prior to use,³⁸ it is good practice to have these systems certified prior to use. The purpose of certification is similar to testing in that it determines whether the electronic voting or counting technology operates effectively. The difference is that an authority independent of the EMB, political parties, the government and suppliers conducts certification. The certification process should be carried out in an open and transparent manner and is intended to build confidence in the operation of the electronic technology.

This certification process will provide independent assurance that the electronic voting or counting solutions meet a certain set of standards. If any changes are subsequently made to the hardware or software, the certification process will need to be completed again, although it may be possible to conduct an abbreviated recertification if changes are minimal and can be categorically identified. Time is again an issue, and the process of certification may take between six and 12 months, depending on how many issues are found that require fixing and how complex the system is. While certification can be a strong mechanism for ensuring the integrity of the electronic voting or counting system and in building trust in the system, it does limit the flexibility of the EMB in making last-minute improvements to the system, as any such improvements would require recertification.

 A number of institutions, such as university information technology departments or technology institutes, could play a role as certifying bodies. It is important that the process of certification is well defined. In some countries the certifying institutions themselves have to be preauthorized and must meet a series of standards for the work they will conduct certifying electronic voting and counting technologies. Clear guidance will need to be developed for certifying institutions on the certification requirements (which should be publicly available), the records they should make of their findings, the consequences of a product failing to comply in some way, the mechanisms for a vendor to resubmit after failing certification and the openness of the certification process and certification reports.

^{37 It should be noted that there are electronic voting and counting schemes designed to provide this level of verifiability for voters (such as Scantegrity, Prêt à Voter and Punchscan voting systems). However, these systems can be seen as quite complex for voters and have challenges in terms of scalability when it comes to larger elections. The crux of the challenge for such end-to-end verifiable voting schemes is to provide verifiability without violating the secrecy of the vote. This is a particular challenge in countries where employers or others could demand that a voter use such mechanisms after the election to prove she or he voted as instructed or where vote-buying schemes could easily be adapted to take advantage of such mechanisms.}

^{38 The Council of Europe (2004) recommendation on e-voting requires that, before any e-voting system is introduced, it be certified by an independent body to verify that it is working correctly and meets all necessary security measures (Recommendations 25 and 111).}

EXAMPLE: Source Code Review Mechanisms in Brazil

EXAMPLE: E-voting Certification Procedures in the United States

KEY CONSIDERATIONS: Testing, Source Code Review and Certification

NEXT:

Election Day (Setup, Testing, Security, Troubleshooting)