ARPA Reporter Usability Improvements
- What I did: UX Design, UI Design, User Research, Wireframing, High Fidelity Mockups
- Client: US Digital Response
- Role: Product Designer
Backstory
Local and state governments, as well as non-governmental entities, who receive federal funding as part of the American Rescue Plan Act are required to report to the US Treasury on how the funding has been used. To address the pain points its government partners were experiencing with the Treasury’s reporting portal, the Economic Stability team at US Digital Response created its own web application for creating expenditure reports.
The ARPA Reporter checks for errors early in the process, something the Treasury’s portal does only after the reports have been submitted and, inevitably, rejected. It also gives users the ability to collect data from sources across multiple agencies and compiles it into the requisite format, saving users time and increasing the rate at which their reports are accepted.
Opportunity
While volunteering on the Economic Stability team, I had the opportunity to regularly meet with agency administrators from six different states who were using the ARPA Reporter. These weekly feedback sessions allowed myself and others on the design team to better understand the administrators’ workflows. In this case study, I’ll go over three feature requests that I worked on during my time as a volunteer. Use the links below to jump to a different section.
Making file versions more informative with upload notes
The primary workflow of the ARPA Reporter tool involves downloading an USDR-provided Excel spreadsheet template (or workbook), filling it out with expenditure data, and then uploading it back into the tool for processing. Afterwards, users can access an Upload Details screen that highlights identifying information pulled from the spreadsheet, as well as metadata about the file, such as an upload timestamp and who uploaded it.
Problem
Several of our admin users expressed the desire to be able to record specific details about uploaded files from within the ARPA Reporter. Since this functionality didn’t exist in the tool, admins were communicating them over email or other methods, which sometimes caused confusion since they were context switching between potentially dozens of uploaded workbooks per expenditure report.
Solution
Adding a free-text comment feature would allow a user to include a note along with their workbook upload. The notes would help create a more informative record of all uploaded files, outlining what may have changed between file versions and allowing others to review the notes to understand the history of a series of uploads.
Creating richer contexts
I started by auditing the existing Upload Details page which revealed an opportunity to improve the clarity and visual hierarchy of the presented file data. By re-balancing the layout of the file metadata and upload series info, I wanted to find space to include notes that would be added to any related files. A few rough sketches helped me arrive at a new layout for the Version History that would be easier to scan and give users a quicker sense of the work being done on a series of uploads.
In addition to the Upload Details screen, I was asked to make changes to the file upload form itself. The team wanted to expand it to include fields for the user to input data associated with the workbook uploads. By setting this information during the upload process, we sought to reduce the likelihood of errors that came with having the users manually typing it into each workbook file.
A basic comment goes a long way
After a few rounds of internal review on my sketches and wireframes, I felt confident enough to put together some low-fidelity mockups to share in the weekly partner meeting with state agency administrators.
Conclusions and takeaways
The project was full of valuable experiences, but the most important lesson I came away with was the importance of following up on my work. I learned this in two unfortunate ways:
- A few months after the feature went live I discovered that while the administrators were finding it useful, some of their staff weren’t using the upload notes and were still relying on email to communicate file changes. Given the opportunity to continue working on the feature, I’d want to interview staff users to understand why and if there was there anything we could do to encourage them to adopt the feature.
- I also discovered the final interface didn’t match the designs I handed over to the engineer responsible for putting them into Production, despite the designs being approved by the Product Lead. Reviewing acceptance criteria as part of the developer handoff process is now a step I have incorporated in my design practice.
Editing the validation status of uploaded workbook files
When staff users upload their expenditure data to the ARPA Reporter, they use a USDR-provided Excel template file. The tool checks the uploaded file for errors like data formatting issues, misplaced decimal points, or incomplete project information. If the file passes, an admin user can validate, or approve, it to be included in the final reporting files sent to the US Treasury.
Admin users validate files uploaded by staff across multiple offices or agencies, and sometimes sub-agencies. When it’s time to finalize expenditure reports to submit at the end of each quarter, the ARPA Reporter compiles all the Excel files together and outputs the appropriately formatted CSV files that are required by the US Treasury.
Problem
During the first two reporting periods of 2023 there were several cases where staff users uploaded an Excel file and admin users validated it but then later determined that the file should not go to the Treasury for a number of reasons. The admin users’ existing options for excluding those validated files from their final Treasury reports were high risk or required support from USDR staff.
Solution
Creating an invalidation process to safely exclude files that were previously approved for final output to Treasury would give admin users more autonomy in the ARPA Reporter. It would also reduce the number of emergency support requests the engineering team would receive during busy end-of-quarter reporting periods.
Establishing ground rules
With the end of the quarter approaching and our government partners needing to finalize their expenditure reports, we needed to work fast to give the admins the new functionality. I started by discussing the feature request with the Product Lead and the staff Design Lead to set a few ground rules:
- Only admin users should have permission to invalidate files.
- Once invalidated, the file would remain in the system since developing the functionality for archiving files would add complexity to the solution. Admin users were also reluctant to permanently delete files because it was common practice to download and externally archive all files related to an expenditure report.
With those parameters agreed upon, we outlined a process for invalidating files with a simple “invalidate” button, which we believed would be a lower engineering lift and allow for a quick turnaround. I set to work making a few low-fidelity mockups.
Starting on the Uploads page, admin users identified which validated workbook file they wanted to exclude from Treasury reporting.
Navigating into the workbook file’s Details page revealed actions that could be performed. When a workbook was invalidated, a timestamp and the user responsible for the action was recorded in the Notes field.
Navigating back up one level to the Uploads page revealed the same invalidation timestamp in the far right column with the cell highlighted in red.
Feedback from other volunteer product designers on my team suggested grouping all file actions together at the bottom of the details table.
Understanding workflows and expected outcomes
I presented my mockups to the administrators in a design feedback session I led which still managed to reveal an interesting fact about their usual workflow.
We learned that out of all the actions available on the Upload Details page, downloading the file was the most frequently used action and the Re-validate function was the least used. This information provided us with an even clearer sense of workflow hierarchy, which I reflected in updated mockups I shared with our engineers.
Up until now I had been designing for expenditure reports that only had one uploaded workbook file in it. However, I needed to expand on my initial mockups to account for the various states an expenditure report could enter if one in a series of associated files was invalidated. In a second feedback session with the admin users, I confirmed the expected outcomes of invalidating files that were part of a series of uploaded workbook files.
Equipped with a better understanding of the admin users’ workflows, as well as how to communicate their effects, we were ready to add the invalidation functionality to the ARPA Reporter for our partnered agencies to use.
Conclusions and takeaways
When the Product Lead brought the feature request to the design team, he proposed we simply “add an Invalidate button” for each file. It sounded almost too easy. But Treasury deadlines and constraints on engineering bandwidth required a straightforward approach to the solution.
The process, from initial discussions to deployment into production, took about 8 weeks. As I wrapped up the project, I noted a couple of usability issues the team might consider tackling in the future:
- Explore Archive or Hide functions to manage lengthy upload lists - According to the agency admins, the number of Excel files submitted with errors could sometimes be high. With the introduction of the Invalidate button, the table view would become cluttered with uploaded files that admins were reluctant to delete, but were no longer relevant to the final report.
- Redesign the Uploads table for visual clarity - Some of the partners found it difficult to spot the most recent version of an uploaded file associated with a project. Highlighting the most recently validated version instead of files that hadn’t been validated or ones that had been invalidated would be preferred and more helpful.
Managing validation template within the application
Before the start of each quarterly reporting period it’s not uncommon for the US Treasury to update their requirements for expenditure reports that state and local governments submit. These updates can range from formatting rules, to guidelines on what data should be included, or even completely new data points. Whenever there is a change to reporting requirements, USDR creates new validation templates to ensure the reports generated with the tool will be accepted upon submission.
Problem
Each time the validation template needs to be updated, USDR engineers need to do so via the backend of the application, which causes the CI/CD pipeline and engineering resources to become a dependency, and sometimes a bottleneck, in unblocking users from generating their expenditure reports.
Solution
Instead, giving USDR admins the ability to manage validation template files from within the application would make the process of updating them more approachable, similar to how one might update content in a CMS. This would allow non-technical, staff admins to make changes that were normally restricted for the engineering team.
Communicating template changes to non-technical users
The feature request had two primary desired outcomes for the admin users to achieve:
- Update the input validation template file by uploading a new version from an interface within the ARPA Reporter.
- View the current input validation template file.
Because the engineering team was normally responsible for certifying any changes made to the validation template file, it was also important for the admin user to be able to confirm that the changes they expected to make would in fact occur without needing help from the engineering team. To tackle the challenge of how best to communicate changes to non-technical users, I created a few wireframes of several options, including:
- Option 1 - Changes are shown as a code block a la Github where changes are highlighted in different colors
- Option 2 - Changes are displayed as a bulleted list, including what metadata is different and where in the workbook the changes were made.
- Option 3 - Changes are displayed as both the code block and the bulleted list.
Mitigating a risky situation
Those familiar with the process of updating template files agreed my initial wireframes covered the basic workflow, but one colleague raised concerns about a blindspot that was missed in early discussions about the feature.
How could we ensure that changes made to the validation template file weren’t going to result in an application-breaking error in the middle of a critical, end-of-quarter reporting period?
Talking through these concerns with the engineering team helped a couple strategies for mitigating risk, which I incorporated into my designs, including:
- A “Restore Previous” button to rollback any changes that resulted in critical errors.
- An additional confirmation prompt before applying changes to add a bit of friction to the process.
Additionally, real-time backend tests to perform QA testing and surface errors to the user were proposed as an idea for a future iteration.
Proposed Solution
As my designs progressed from initial wireframes to high-fidelity mockups, the most pronounced difference was in how the current validation template was displayed. After learning that changes made by the validation template occurred in two locations in the application, I divided the code block into Environment and Template Rules, which mimicked how the template file was displayed in the application’s Github repository.
By this time, the design team had already made great strides in building a new design system for the Economy Stability team’s web applications and I was excited for the chance to apply it my final mockups.
Future Considerations
The template management feature was quite the rewarding project to end my volunteer rotation with because it also solved for managing both the input template file used to validate the individual workbook uploads submitted by staff users (read more about these in Challenge #2), as well as the output template file used to validate the final expense report output by admin users.
I was please to hand off my finalized mockups knowing the design was being added to the team’s ARPA Reporter product roadmap. Given the opportunity to continue volunteering with the team, I would have looked forward to:
- Testing whether the added friction of the two-step confirmation process actually worked and if it was an effective method for avoiding unwanted changes.
- Revisiting the idea to implement a live QA process to surface errors in real time. Figuring out the best way to display those errors to users would have been a fun challenge and new tests could be written to better mitigate critical errors based on past attempts to change the validation templates
Conclusion
Volunteering with USDR as a product designer was one of the most rewarding experiences I’ve ever had in my career. I further developed skills and strategies for working cross-functionally. I gained a new appreciation for user research and speaking directly to users in order to better understand their needs and workflows. Lastly, I learned that even changes to small components can have an impact on the overall experience and usability of a product, and how that can affect user acquisition and retention.
I felt so humbled by the knowledge and experience that was eagerly shared with me by the other designers on the team, and was inspired by the many talented people, both staff and volunteer, who were working to make a positive social impact. If you're interested in lending your talents (even if they're not design-related) to a mission-driven nonprofit, consider "raising your hand" as a volunteer on their website!
