OdysseyMate

During the process of testing OdysseyMate, I learned a lot about testing, and had a lot of thoughts on things I wish we could have done before testing, and things we still have left to fix after. The first thing that I learned was that instead of describing the gist of the task in the task list, we perhaps should have just numbered them. Some tasks required users to reach a certain screen before they could do the main point of the task, but when users read things like “Task 2: Create a new pack”, they are inclined to just start doing that before reading the steps in the bulleted list below. I also learned that prototyping app screens gets increasingly difficult with an increasing number of built-out features. While the screens were necessary, I wish I had stayed more organized when connecting them, because prototyping is a bit of a nightmare now as everything looks like a giant spider web. Additionally, while prototyping, I wish I had clicked around more randomly to simulate what a typical user would do. It was a bit painful to watch people click on the wrong item and get redirected back to their California trip page by hitting the home button mid-task, because we had to interrupt to return to the correct page. I also learned that if a question arises early, you can’t brush it under the rug, or it comes back. For example, we received a question about how guides would be rated early on and I ran out of time to implement all of the screens required to make it make sense, also I figured users wouldn’t end up interacting with a guide rating page in either of the task lists regardless. However, one of our testers asked a similar question again during their post-test interview, and was bothered by the idea of not being able to determine the validity of a 5-star review. Another group during our heuristic design pointed out that the packing list might be a bit small. We ended up enlarging it slightly, but it still proved to be difficult for testers to read and interact with, so we hadn’t completely fixed the issue in our first revision. I felt that having a 2 person group test was the best part of testing the application, as they spoke more freely than our other testers and had a lot of interesting feedback during their post-test interview. It was also helpful to have them test first, as they were able to assist each other in completing tasks when one of them got stuck on a page. This had the added benefit of knowing what was fully detrimental to testing if it was unfixed, because if 2 people couldn’t work out what button to press after an extended period of time and talking it over, we doubted individual testers would be able to figure it out alone. I also felt the post-test interviews went well, because some testers didn’t speak much about the application at all while clicking through it, so the interview was necessary to get meaningful feedback. I also learned that some testers catch on to how Figma works much faster than others, which can skew test results. In hindsight, we should have researched to try to figure out a way to remove the flashing buttons when users click on the wrong area, because some users seemed to understand from their first few mis-clicks that the app was telling them where they needed to click to continue on with the task. This made it a bit difficult to analyze the number of mis-clicks, since some users seemed that they were able to get through tasks just by following the flashing pointer. I was also glad we recorded because it made capturing the number of mis-clicks easier, and it was too difficult to keep track of everything in one watch. Overall, I enjoyed the testing and learned a lot for if I conduct a usability test again in the future.

User testing in my opinion was pretty good for the most part and I felt like all of our users were able to get through the given scenarios and tasks. The biggest take away and learning experience from user testing that I learned was that users will find the loopholes in the prototype and often navigate to them. So having a robust and really well tested system/ prototype before even getting to the user test is crucial; not only for testing, but for the tester. Unexpected things that occurred during the test that I noticed that happened a lot was that users skimmed the scenarios and tasks that were written down and were more focused on interacting with the prototype. This caused confusion for a large portion of our test users when running through our second scenario when they were tasked with creating a new pack in the Anaheim, CA trip that they had just created in the prior task. Things that could’ve gone better for me is have a little bit more of a robust prototype or slightly more refined one. There were a couple times when testing that the user got to pages / screens where they should not have been, so we have to reset to the correct page or to the last page they were at and had them run through the task again. I think additionally if we also refined the scenarios and tasks, it would’ve maybe helped with causing less confusion. Additionally, limitations of Figma I felt detracted from the experience of some users. User testing taught me a lot about prototyping, design, and how outside users unfamiliar with your system interact with your system. As the designer of your own system, you think that your system is intuitive and easy. But when given to users to test, it is often not the case. User testing has really helped refined our system for which I don’t think we would’ve caught had we not done user testing.