Web Analytics

The web analytics assignment made a good part 2 for the usability testing project. Our group was able to make use of much of the information we gathered in previous assignments and create a plan to further test it using web analytics. We first analyzed attributes of the site and its users that we were interested in. We chose to test the length and depth of visitors' stays on the site.

The data from analytics supported our thoughts that very few users stuck around MemCatch for very long. Both number of pages viewed per visit and overall time of visit were very low, with most users only visiting a couple pages before leaving and very few sticking around for an extended period. We used this information to come up with a possible plan to get users to stay longer. We thought that we a more intuitive way for users to navigate the site would do the job. After designing a rough outline of our plans, we decided on which measures to observe as we tested them.

I found this assignment to be very educational in giving a way to act on the observations we have made in other assignments and the usability testing assignment in particular. Though unrealistic, it would have been nice to see the results of such an experiment. Even so, the knowledge gained from the assignment would be very beneficial to anyone looking to make use of information gathered from observations and tests.

Usability Testing

From the original description, this assignment seemed like it would be a lot of work but one that could give a lot of beneficial experience if enough time and effort were put in. My prediction was correct. During the weeks that our group worked on the assignment, this were very hectic and difficult but the experience was very rewarding.

Some of the difficulty came from the nature of group work and the problems that arose from very different schedules. One member was out of town while the bulk of the work was to be completed and another member was very busy planning another event. Finding time when all of us could meet to conduct the tests became impossible and our first draft suffered for it.

Fortunately, we decided to re-do much of what we had come up with in order to make things right the second time around. We wanted to make the project worthwhile so extra effort was needed. In our second set of tests, we gleaned much better information and were much more productive. Even with all three members present, there was a heavy workload even after completing the tests. Our discussion and analysis of the tests took a lot of time.

From our experience I would bet that usability tests are an expensive option, especially on a per-participant basis. Setting up the equipment took time and patience, planning was very difficult, and analysis took a long time to complete. Upon completion, the benefits were very noticeable. In comparison with other methods we have discussed and used, the usability tests gave the most thorough information about how a potential user would interact with the website. For companies that have a working prototype or beta version, usability tests would be indispensable. In no other way could one get a more accurate view of a user's ability to navigate through a piece of software. Capturing the mouse movements and clicks as well as a user's expressions makes the data very easily to go back to and to quantify. The presence of experimenters taking notes and a moderator also makes for accurate information gathering.

For software in a stage such as that of MemCatch, usability testing would be a very valuable resource to make further improvements. It allows for developers to calibrate their thoughts on the needs and habits of potential users while also finding potential bugs and hurdles in their current system. With the proper sample and analysis of data, usability testing could make a huge difference in the outlook for a company like MemCatch.

Persona/Scenario

Creating personas and scenarios is a very important part of designing a website, both for form and function. They give developers a window into who might be interested in their site, and whom they need to cater to.

I found this assignment to be somewhat challenging to complete. The subjects we were supposed to model personas after (from the interviewing assignment) were not very diverse. Being allowed to deviate on one or two of the three assignments was helpful, but I still found it difficult to come up with truly diverse potential users of MemCatch. Perhaps I do not have a full understanding of whom MemCatch is marketed toward but I did give it significant thought. I decided to base two of the personas on interview subjects and create a third completely on my own.

I tries to create scenarios that were closely tied to the uses and personalities of each user. I also tried to create varying uses and experiences for each persona. This was also somewhat difficult due to the similarities in personas mentioned earlier. I also further investigated MemCatch to refresh myself on features and see if there was anything new or different from the last time I was on the site. My goal was to create scenarios that utilized as many of MemCatch's features as possible. I wanted my personas to use the full breadth of the site.

Despite the difficulty and frustration, I found this assignment to be a good learning opportunity. It forced me to learn more about the site and expand my understanding to what other users might see and find useful. Honing skills like that will be very useful in future work with software and user experiences.

Heuristic Evaluation

For this assignment, we were required to perform a heuristic evaluation of our website. I had a few ideas of what I would address in MemCatch from previous visits to the site but actually going through and looking for issues produced many more results than I would have thought about otherwise.

When going through the evaluation I simply noted what went wrong and how it affected my experience in a list of observations. The list was very rough and did not give much detail or information on the problems. This made it much easier to quickly navigate the site and quickly come up with a large list to further evaluate later.

I started the report without much of a hitch. However, I noticed an issue a few days later when I tried to recreate problems. Many significant portions of MemCatch had been redesigned and some of the issues I had already raised had just been solved. I adjusted my report accordingly, adding some other problems and noting problems that had already been solved. In some ways I was upset that my evaluation would be useless, but I was glad to see that MemCatch had made beneficial changes. The redesign also attested to the legitimacy of my complaints (if only to boost my own confidence).

Interviewing

I found this assignment to be useful as practice in creating and implementing a questionnaire for a potential user of MemCatch. I worked on a team with Maria Lee and we were able to collaborate very well in all parts of the assignment. I enjoyed the opportunity to read the questions of another group and receive feedback both from them and the professor on our potential questions.

The other group we consulted with was able to give a fresh perspective on what our users might be like and what kind of questions they would work well. When I was coming up with questions, I let my own knowledge of MemCatch affect my thought processes and the knowledge required to answer my questions. The other group was able to tell us what would make sense to someone who was unfamiliar with the site.

I thought the interview went very smoothly and that we were able to gain a lot of information from it. Our interviewee was very helpful in her responses and she was very descriptive as to her common knowledge gathering practices and goals. Because of the helpfulness of some of the responses I was unsure as to whether our questions were just worded that well or whether the interviewee was just trying to give generally useful information.

The interview took about as long as we had planned and did not exceed one half hour. After transcribing the whole interview we were able to go through and see what was relevant and what we could use to capture a general persona of a MemCatch user. Fortunately, the interviewee possessed many of the qualities we hoped to find for a potential MemCatch user. This made the report process much easier and we were able to make connections between her goals and the functionality of MemCatch.

The interview assignment went very well overall and I feel more confident in my ability to gain information from potential users and customers in a one-on-one setting. Maria and I worked very well together and we were able to collaborate in all aspects of the assignment while working efficiently. The responses from the interviewee even helped me gain a better understanding of MemCatch by exposing me to a different set of needs that the site seeks to accommodate.

Competitive Analysis

I found the competitive analysis assignment very useful in giving perspective to the type of site MemCatch is and what it offers. I spent a lot of time navigating the MemCatch and competitor's sites in preparation. This gave me a much better feel for what the primary goals of the sites are and what the important functionality is that lets users accomplish those goals. After getting acquainted with each website once, I then went back through two or three more times to look for features I found in the other programs and find ways to accomplish similar goals.

With one of the competitors, a primary problem was that the service was made primarily for organizations and was not available for individual use. The site did have a useful instructional video that went through features and gave demonstrations. This allowed me to gather how one is meant to use the service and actually accelerated the learning process for that program. Instructional videos seem to be a great starting point in choosing and learning how to use a particular service. I was glad to see that MemCatch is preparing a video for their website in the future.

Analyzing competition was very useful to me in getting a feel for what MemCatch needs to be successful. It is a great way to help prioritize important features and accumulate ideas for additional functionality. I found that MemCatch could definitely benefit from additional features that I might not have thought of if I hadn't looked at a few competitors. Methods of organization also seemed more intuitive in some places in the other programs. By reorganizing items in a similar way, MemCatch could greatly increase user's ease in learning how to use the program.

Overall, the competitive analysis seems like a very cheap and relatively quick process of finding new goals and ideas for improving one's product. In most cases it seems like finding a competitor to compare to shouldn't be too difficult or costly. A product can gain a lot by looking to competitor for information on what does and doesn't work.