Testing Data

Does your organisation develop software in house? Are you responsible for testing applications? Are you a tester of applications? If so, where do you get your test data? You know, the data you put in the databases that allow you to test the applications, where does it come from?

If the answer is from the live databases then you could be in trouble. Yes, you personally.

Many organisations, both large and small, take cuts of live data to use for testing purposes. Indeed, this is often seen as a good way of generating the test data required to fully exercise the application code. But is it?

There’s no argument that it’s as close to ‘live’ as possible because it is live, but do you do anything with the data before you place it in your development and test databases? Do you sanitise or obfuscate personal or sensitive fields? Do you restrict access to properly vetted and checked staff? If not you could be in breach of the law or regulatory conventions governing your organisation.

In the UK, personal information stored electronically comes under the Data Protection Act. Any such data collected can only be used for the purposes it was collected for. If ‘testing our application’ is not included in the official purposes, then raw personal data taken from the live databases and used for testing is almost certainly done so in breach of the DPA.

In addition to the DPA, which covers personal data held on individuals, there are other good reasons for not taking live data, unaltered. These could be regulatory or compliance related, including FSA, PCI, or any other governing regulations.

But it’s not just about the rules and regulations. It’s just really bad practice. If all you’re doing is testing your application against known, good data, then you’re not properly exercising your code. You’re not performing thorough data and error handling tests. Part of application testing is ensuring the code handles the unexpected in a predictable and safe manner. If you don’t test with data that stretches your code, how do you know what your application will do should it come across such data? 

So what should you do? How should you populate your test data?

This is where it gets interesting. These are good and valid questions, and there is no easy answer. The simple fact is that you are going to have to spend some time and effort on this problem. Test data needs to be created or generated and it needs to be good for testing. Taking a cut of live data and performing careful and thorough obfuscation is obviously one way of doing things. It needs to be done carefully though, as sensitive data may not be immediately apparent, for example, some free-form fields may contain sensitive data even after obfuscation. It’s also no good just populating the database with meaningless random data, as the testers may not be able to tell if they have the right results.

So it’s likely that the only way to achieve meaningless but realistic test data is to take the time (and money) to generate it, utilising relevant resources (DBAs, testers, developers, data owners) to ensure it covers the testing requirements. This test data could then be stored, added to, modified and reused throughout the lifecycle and lifetime of the application. You could even set up a function to centrally manage and distribute test data for all your applications and data users, ensuring high quality, useful and valid test data is available when needed.

There are a number of tools on the market which can aid with this process. A simple Google search for “database test data generator” pulls up dozens of suitable products out of about 1.7m hits, so there are resources available.

At the end of the day it comes down to a conscious decision by your organisation. You can either continue using live data for testing purposes in the hope that no security breach will occur, and that your breaking of the law or regulatory policies won’t be discovered, or you can stop cutting corners and take the time and effort to correct the situation.

Remember, ignorance of the law is not a defence under the law.

Article reproduced with kind permission of Pentest Limited - www.pentest.co.uk

--