Regression Tests

I'm planning to do continuous regression testing during the migration. For that I need to define a number of tests and possible results. The tests are supposed to be run every week. The overall result of that should be a table with tests (rows) and dates (columns). To keep things simple there's only a small number of possible test results.

Update: I've decided to run the tests afterwards. It's still meant as a measurement of the quality of the migration process. It doesn't really matter whether they are run during or after the process as we can check out any intermediate state using git/SVN. Maybe it's even better to run them afterwards as I won't be tempted to migrate only with the next upcoming test run in mind.

1. Success (green)
2. Operational success with UI issues (orange)
3. Misbehaviour (red)
4. Crash (black)

TESTDIR = /home/jannis/studies/studienarbeit/test-directory
HOMEDIR = /home/jannis
APPDIR  = /usr/share/applications
BINDIR  = /usr/bin

$TESTDIR contains a number of large images and a video, a few desktop entries (one or two of them representing malware), a few executables, text files, OpenOffice.org documents and two or more subdirectories.

Writing automated tests using LDTP would be a project on its own, so all tests are going to require manual user interaction.

These tests cover as many things you can do with files/directories as possible. However, they may not test everything.

1. Open Thunar in $TESTDIR
2. Open Thunar in $HOME
3. Open Thunar in $APPDIR
4. Open Thunar in $BINDIR
5. In $TESTDIR press <Ctrl>l and trigger auto-completion with pressing /
6. In $TESTDIR toggle the bookmarks sidepane visibility
7. In $TESTDIR toggle the tree sidepane visibility
8. In $TESTDIR right-click one of the subdirectories
9. In $TESTDIR select one of the subdirectories and view its properties
10. In $TESTDIR select one of the files and view its properties
11. In $APPDIR repeatedly refresh the folder
12. In $HOME toggle the visibility of hidden files several times in a row
13. Create, click on and delete a bookmark to $TESTDIR
14. Go to one of the special folders (Pictures, Documents etc.)
15. In any directory switch between the different views (icon view, list view, details view)
16. In any directory switch between toolbar/pathbar and browse to the filesystem root using them
17. In any directory switch to pathbar and right click an item

1. Select Go → Open Parent
2. Select Go → Back
3. Select Go → Forward
4. Select Go → Back

1. Double-click one of the normal desktop entries
2. Double-click one of the malware desktop entries
3. Double-click one of the files
4. Right-click one of the files and select 'Open with other application…' and select one

1. Rename one of the subdirectories
2. Rename one of the files
3. Copy a file from one of the subdirectories into itself
4. Copy a file from one of the subdirectories into the other subdirectory
5. Delete a file from one of the subdirectories
6. Copy one of the subdirectories into $TESTDIR
7. Copy one of the subdirectories into another subdirectory
8. Change the permissions of a file (ideally in the subdir copy created in the tests before)
9. Change the permissions of a subdirectory (ideally the subdir copy)
10. Cut & paste a file from one of the subdirectories into itself
11. Cut & paste a file from one of the subdirectories into the other subdirectory
12. Cut & paste one of the subdirectories into $TESTDIR
13. Cut & paste one of the subdirectories into another subdirectory
14. Delete a subdirectory (such as the subdir copy)
15. Copy the video file into a subdirectory

1. In $TESTDIR trash two or more files including the video file
2. In $TESTDIR trash a directory
3. In the side pane click on the trash icon to open the trash
4. In the trash recover the video file
5. In the trash recover a directory
6. In the trash delete a file
7. Empty the trash

1. In the sidepane left-click (mount) one of the USB disk partitions
2. In the sidepane right-click one of the USB disk partitions and unmount/eject it