Figure 1: NY Philharmonic Concert in Central Park. Source: NY Tourism
What is NY Philharmonic & What is This Project About?
Founded in 1842, NY Philharmonic is one of the oldest philharmonic orchestra worldwide. Based at the David Geffen hall at Lincoln Center, it appeals music professionals and enthusiasts from all over the world to appreciate the art of classical Western music. Recently, NY Philharmonic’s digital archivist & digital asset manager, Bill Levay, have partnered with Pratt Institute’s Center of Digital Experiences to ask for a few user experience consultants to help test and provide feedbacks of the newly developed NY Philharmonic Archival Website. Bill Levay was concerned that whether this new archival website need any improvements in accessibility, such as whether the structure needed to be simplified or certain functions needed to be modified or reorganized to ease the process people use the website. Bill Levay wanted to understand how different types of users interact with the new digital archives interface before its public launch.
Figure 2: Newly designed NY Philharmonic Archival Website waiting to be analyzed
I, together with two other user experience consultants, Megan Fitzmaurice and Carlos Soriano, were responsible for carefully analyzing the website, moderating client interviews, receiving feedbacks, generating high-fidelity mockups as solutions in improving the accessibility of the website, writing final report, and presenting the findings and solutions to the clients.
- UX Consultants: Megan Fitzmaurice, Carlos Soriano, Tommy Tian
- Course: INFO-644 Usability Theory & Practice
- Professor: Craig MacDonald
- Tommy’s Responsibilities: Moderating client interview sessions, providing feedbacks based on the interview sessions, and organizing executive summary, conclusion, and appendix for the final report
- Time Lapse: April 2026 – May 2026
Getting Set Up for User Testings!
We set up our moderating platform, Private Panel, with our Pratt Institute emails and we added the clients into our Private Panel session called Ny Philharmonic. Due to our clients’ concern of privacy, we will not show their full names.
We have also determined what types of clients we will be interviewing with:
- Profile 1 — General User
The category “General User” would have casual interest in music, history, or cultural content; no prior experience with digital archives or library databases is required and using websites regularly for general browsing is accepted. However, they are required to use the internet regularly.
- Profile 2 — Researcher / Expert User
The category “Researcher / Expert user” would have experiences working in or studying music, history, journalism, or a research-oriented field, they are also required to have experience in using a library database, archive, or similar research tool at least once. They must also be actively engaged in academic, professional, or journalistic work involving historical sources.
Figure 3: The Private Panel session of NY Philharmonic
Based on the website’s structure and contents, we then created the tasks for our clients to complete during the future moderated sessions. The tasks are:
- Task 1: Search for programs & scores, create an account, add to cart
“You’re interested in the New York Philharmonic’s history with American composers during the 1940s. Find a concert program from that decade that includes a performance of a work by George Gershwin. Next, find a Gershwin score with a corresponding part set and make a request to download both the score and a timpani part.
- Task 2: Performance history and CSV export
“You’re writing an article about the popularity of certain symphonies. Find out exactly how many times the New York Philharmonic has performed Beethoven’s Symphony No. 5 throughout its entire history and download the search results as a CSV file.
- Task 3: Search for business documents, add to cart
“You are researching former Music Director Pierre Boulez. Find and request to download a piece of correspondence (like a letter) and a photograph of him conducting during his time with the orchestra in the 1970s.
Before we send the task to our clients, we first need to make sure they are understandable and completable, so we did pilot test with some of our friends to make sure the tasks can at least be understood and operated. I did my pilot test with my friend Andy, who is a student from New York University studying electrical engineering, who also had some experiences in data collection and analyzing. Andy has authorized me to use his name.
Figure 4: Me asking Andy to complete the pilot test
Figure 5: Andy responding the pilot test
Andy told me that all the tasks are understandable and completable, meanwhile Megan and Carlos also reflected that their friends said the pilot test worked for them.
We sent the questions to the clients Bill Levay recommended to us since he is the person who asked us to help him gather feedbacks. Bill Levay recommended some music researchers to us whereas Me and Megan found some friends who have experiences in music and archiving. Megan found her classmate from the Library Science major in Pratt Institute, whereas I found my friend who plays rock music and owns a band, Simon. Simon has also authorized me to use his name. In total, there are seven clients who accepted our moderating request and all of them gave permissions for us to record the zoom meetings while moderating with them.
Megan and Carlos were struggling finding six clients because some of the clients did not respond at all, which might cause our group to not reach the minimum requirement of client numbers; but after I invited Simon, we got six clients and got enough people to receive feedbacks from!
Before we started the moderated sessions officially, we asked our clients to answer a pre-test questionnaire:
Figure 6: Pre-test questionnaire, questions 1-3
Figure 7: Pre-test questionnaire, questions 4-6
All of the seven participants answered the questionnaire, which gave us the appendix below. We made names and emails anonymous to protect our participants’ privacy.
Figure 8: Appendix of pre-test questionnaire
Based on the pre-test questionnaire appendix, all of the seven participants are at least somehow interested in music, and they are all comfortable with using searching engines. The major differences are the different occupations of the participants and their different experience in digital archives.
Last but not least, to make sure our participants give us consents of accessing their voices and faces during the zoom moderated sessions, we kindly asked them to sign a consent form before the moderated tests. Fortunately, all of our clients have agreed to give us consents.
Figure 9: Consent form
Moderated Sessions: Listen and Receive Feedbacks!
From April 20th to April 29th, Me, Megan and Carlos conducted the moderated sessions with the participants. The steps of conducting a moderated sessions are: me, Megan and Carlos will divide into moderators and scripts, moderators are responsible for asking the participants to try their best to complete their tasks while scripts are responsible for taking notes, recording the meetings and providing transcripts.
I was responsible for moderating two sessions while being the script for both some of Carlos and Megan’s sessions. We organize the recordings and notes into folders according to date and participant.
Figure 10: Our moderated test folder
For example, here is one of the sub folders containing my contents as a script, including: audio of the meeting, video recording of the meeting, notes and scripts. The scripts were generated with a transcribing software called Clipto.
Figure 11: A sub folder containing notes, script, and the recording of moderated sessions.
Some interesting quotes and behaviorshave appeared in the two sessions I moderated:
- Participant 1: “Does it matter whether it is from the point of view of a developer?”
Participant 1 has told us that he is a web developer besides a music enthusiast, and since he has web development experience he can be very precise on what are the problems on a website. For example, he criticized the filter is very long and very slow to load, however when people use websites, most of them would demand efficiency, which means less waiting and scrolling time; he also pointed how it is hard for him to find the exact matches on the website, but people use website to get the answers/results they demand. Participant 1’s notable behavior was how he criticized the website was slow, he used a kind of joking tone to say “These filters are taking me lightyears to load”, I understand he does not want to make me feel awkward but I understand the website still needs a lot of improvements… - Participant 2: “I know what my jobs are, but I just don’t know how to do it.”
This is by far the most harsh comment I have seen on this website and I am sure Jerry felt the same way too, both participants plays music a lot, however, because of the messy design of the website, they cannot find the proper way to complete the tasks EVEN THOUGH they understand what the tasks mean. For example, Participant 2 complained that he cannot find the scores because on the website there are no direct indications about where the scores are located. Participant 2’s notable behavior was he struggled a lot finding where to search for the keywords for the tasks, her looked very confusing, clicked everywhere on the website, and in the end he has to say “I don’t know what to do” or “I give up”.
Even though the pilot test was successful with my friend Andy, we noticed there were still a lot of issues while our actual participants were trying to complete the three tasks. I also looked at Megan and Carlos’s sessions while they looked at mine. In general, all of the participants complained lack of download function, hard to find how to search for the exact documents described in the tasks, and a long filter which made them complain hard to read through and hard to find the correspondent search results. We have included those feedbacks in our final report:
Figure 12: Detailed feedbacks for each tasks
We then focused providing recommendations and high fidelity mockups of the solutions. Megan was responsible for creating the high-fidelity mockups while me and Carlos were responsible for providing recommendations.
I was responsible for generating multiple recommendations and they were all adopted and included in the final report, and here they are:
- Tommy’s recommendation 1: Clarifying Date Filter Labeling
Figure 13: The original date filter with problems described in the screenshot
Figure 14: modified date filter with descriptions of improvements
- Tommy’s recommendation 2: Streamline top navigation section to clearly distinguish between Performance History and Digital Archive search.
Figure 15: The original top navigation section with problems described in the screenshot
Figure 16: The modified top navigation section
- Tommy’s recommendation 3: Change the “Add to Cart” function into “Download Request”
Figure 17: The original “Add to Cart” button, which is confusing for our participants
Figure 18: Changed to “Add to download request”
Final Presentation and Conclusion
After finishing the final report, we summarized report into bullet pointed and created our final presentation slides. We conducted our final presentation during the last class, which included the test results, feedbacks, recommendations, solutions, and conclusions to Bill Levay.
Figure 19: Our final presentation, full link can be found here
We created a folder for the full presentation with screen recording, transcript, and the slides.
Figure 20: Folder of the presentation deliverables
Figure 21: Photograph of we three doing the final presentation
We then finished the final draft of the final report and sent it to Professor MacDonald and Bill Levay. The group also conducted its last meeting to give each other feedbacks and conclude the results. I was praised for providing high-quality screen recordings and precise transcripts, and I was also proud to have three of my recommendations on the final report; I expressed gratitude to Megan and Carlos for accepting my recommendations and I am glad that compared to former group projects, more of my recommendations got accepted and I began to contribute using the abilities that I am truly professional at, such as attention to detail, providing feedbacks based on details, and file organizing.
In the future, we hope to hear back from Bill Levay about how the Ny Philharmonic management team think about the final report and the presentation, and we would be happy to see the improved website and if possible, providing more feedbacks and improvements as they publish the website worldwide.