CAPTCHA Testing

Functional Usability Accessibility

  When the CAPTCHA is not entered, there should be a client side validation which displays error message on submission of the form.

  CAPTCHA entry should be case sensitive.

  All the data of the form fields should be retained if error is occurred for CAPTCHA text field.

  There should be a link (AJAX) which reads like, “Refresh the text in CAPTCHA”. That would help end-users in refreshing the captcha characters because they were not comfortable in understanding some characters in the current image.

  There need to be audio support if the website is supported even for blind end-user or even partially blind end-users.

  Too much background noise might even spoil the comfort in listening to audio captcha. It is good to consider opinion of audio captcha from many people. Considering partially blind and blind people to listen to it might be an awesome idea.

  If web service is being used and is fetched from another server, it is important to see whether it is rendered smoothly in sync with the other form components and web page elements. In my experience, I have seen captcha being loaded after few seconds, after all page elements are displayed which gave me a feeling that there is nothing to be displayed and whole thing is completely loaded. Then, later suddenly it displays giving an odd feeling.

  Proper TAB indexing should be done even for captcha text field. I have experience where reCaptcha was used in registration form and TAB indexing was missed for it. Then I suggested to the developer to fix that as there is option by reCaptcha Google to provide tab indexing option.

Security

  CAPTCHA images should not reveal absolute path names. Usage of web services is a good idea, just like reCaptcha.

  Do not have cyclic fashion captcha images. Like 1 to 100 and then again 1 to 100. Easy to crack. It is good to have some algorithm which generates huge number of captcha images using image libraries.

  Usage of background noise in the image, different textures, and different angle of displaying the characters might be a good idea to make it difficult for some captcha cracking programs like http://free-ocr.com/ and few others.

  Audio to text converters – Use some of these software(s) and see whether they are able to crack the audio captcha or not.

  CAPTCHA should refresh on every wrong entry. Keeping it static might be vulnerable to brute force attack for captcha to bypass it.

  There needs to be server-side validation for CAPTCHA entry. Use Firebug to Inspect Captcha element and then just delete it from client-side. Then, just fill the form without captcha and submit it. If it gets submitted, then there is no server-side validation which is a high risk one. It’s equivalent to not having captcha.

  CAPTCHA with question and answers in plain text and mathematical functions questions in plain text are not recommended in my opinion.

  Combinations of uppercase / lowercase alphabets, numerical, special characters could be used to increase the brute force combinations for CAPTCHA which would turn out to be very difficult to crack CAPTCHA quickly. Hackers usually do not employ brute force for so many numbers of combinations; rather they would hire a human to bypass the captcha manually. Well, yes. There are CAPTCHA breaking services.

  Saving list of questionnaire for CAPTCHA in JS file is easily vulnerable as all the questions could be retrieved easily and assertions could be easily added using some automation tool like Selenium and bypass CAPTCHA. I had seen this vulnerability in check-in service web application Gowalla or Foursquare – I do not really remember which one exactly.

Testing Experiences By Santhosh Tuppad

 I had hacked the captcha by getting absolute path names. The images were named like 1.PNG, 2.PNG, 3.PNG…200.PNG. I used DownThemAll and gave range to download all images. Then I just prepared text equivalent of those captcha’s within 1 hour of time and had all 200 captcha equivalent text for those. Then added assertions like, when 1.JPG, enter the corresponding text. Then I was successful in bypassing any captcha.

 This was for Gowalla or Foursquare (Again I do not remember exactly), I was able to remove the CAPTCHA component from client side using Firebug and then submit the registration form without captcha. Guess what? I was able to register successfully. So, server side validation is a MUST.

 In Mozilla Firefox quality.mozilla.org, I saw a Turing test which always had a same question which means same answer. It was surprising to me. I reported it to them and now it is fixed.

 One of the government websites of Indiahad a CAPTCHA which was easily cracked by using http://free-ocr.com/ — I cannot reveal the website name because it has not yet been fixed.

References

 http://en.wikipedia.org/wiki/CAPTCHA

 http://www.captcha.net/

 http://caca.zoy.org/wiki/PWNtcha