Role
Product design
Team
Product Manager,
UX Copywriter, Engineers (x7)
Timeline
October-November 2023
Upwork is a marketplace for freelance work that matches quality talent with clients on a variety of project spaces.
Nearing the end of 2023, the topic of Circumvention became increasingly important, as it would become the company's number one priority for the following year. Through prior research as well as more current generative research, we discovered that there had been a significant increase in contact information being shared between the Client and Freelancer prior to the start of a contract- indicating a higher likelihood of circumvention.
In this context, circumvention is defined as business being taken off the platform, to avoid any fees incurred by using our services. When this occurs Upwork is unable to provide safety measures to ensure proper payments between parties, scam detetction, and more. Additionally, from a business perspective, platform circumvention has been estimated to cost Upwork between $325-333 million in gross sales value, annually… Because circumvention occurs off-platform it is often difficult to measure.
In October of 2023, my team and I would be tasked with addressing the initial phases of Circumvention, with constraints inclusive of a 2-3 week turnaround time.
We do not provide enough education to our customers associated with the risks of taking their conversations outside of the platform, nor inform them of the benefits of staying on the platform.
We do not inform our customers adequately about the impact of sharing contact information directly.
Customers are often unaware of Upwork's anti-circumvention policies or the consequences of violating them. They are also frequently unaware of how to identify and report attempted circumvention.
Business Goal
User Goal
How do we ensure our customers are aware of our anti-circumvention policy while reducing the likelihood of it occurring?
Research Overview
As we began to conduct research, anonymous surveys became a large part of this research as sensitive questions about past circumvention attempts would be asked.
We anticipated that customers may be fearful in being completely honest as circumventing the platform may often result in being banned from using the platform…however we anonymized it ensured we would not seek out repercussions.
Due to time constraints on this project, we did not get the opportunity to conduct 1:1 interviews as I would have hoped to do. Much of our research findings came from prior analysis.
Client v. Freelancer
For both Freelancers and Clients, customers experience confusion around the circumvention policy. However, freelancers expressed more pressure to communicate off the platform for fear of losing the job opportunity.
Takeaways
Customers were unaware
of what was considered circumvention
FLs felt pressure to
communicate off platform
if asked by the CLs
FLs felt pressure to communicate off platform
if asked by the Clients
Circumvention User Journey
The areas most likely to circumvent the platform occur prior to a contract being created, and attempting to end a contract early.
Throughout the Journey we have done our best to educate both Clients and Freelancers about circumvention and bring awareness into the conversation, however have noticed key moments where there could be improvements.
Our key strategy for benefits encompasses benefits such as assisting with payment disputes, added levels of security, and protection from scam, fraud, and abuse.
The Solution
In meeting with our cross-functional partners and stakeholders, it became apparent that we would need to introduce a "hard block" at the beginning of the user journey in the instance that either the Client or Freelancer attempts to share contact information before starting a contract. We would continue to block this information up to three times, and on the fourth attempt, we would allow this information to be shared.
This would enable us to more easily keep track of customers who were intentionally circumventing the platform, and allow us to study this behavior following this initial launch.
Iterations & Cross-Functional Collaboration
Following further collaboration with Legal and Content Design, the team narrowed our designs down for when a user decides to end a contract early. We were unified in our understanding that the goal here was to educate our customers and allow for them to make informed decisions. At Upwork we have "fixed-price" contracts and "hourly" contracts, therefore it was important to identify the differences of each. Our Enterprise product would be included in these as well.
Because this project was only 2-3 weeks long, we moved directly into high-fidelity, utilizing our extensive design system.
Measuring Success & QT Results
With such quick turn-around time on this project, we were able to learn very quickly from this release.
We found that contact information shared prior to the start of a contract decreased significantly, down to only 1%.
45% of customers (both CL and FL) attempted to sent contact information twice before accepting the circumvention policy and deleting their message.
4% of customers (both CL and FL) moved past each attempt (3) to block the sharing of contact information and ended up sharing it regardless.
Post-MVP Alternatives
Following up from our surveys, we discovered two key items we could take into consideration moving forward, and proposed three suggestions to encourage customers to report circumvention attempts.
Clients aren't afraid to report.
When the client is the recipient, we could encourage them to report the FL.
Talent is far less likely to report.
When the talent is the recipient, we can empower them with a suggested way to reply to the client.
Or we could take an empathetic approach…
Even if contact information is shared by the talent, we could let them know that we are aware they may sometimes be pressure into sharing contact information and empower them with the same or similar suggested reply.
In the process of adding to case study…click below for presentation
GenAI
GenAI has the ability to act as an additional resource in filtering out reputable talent.
With the introduction of AI to Upwork in 2023, there was room to seek further opportunity for Trust and Safety improvements.
What do we want to achieve?
Expand AI efforts to allow for higher-quality talent with the ability to flag inappropriate content via Natural Language Processing (NLP)
Allow for GenAI expansion
via cover letters
Expand AI efforts to allow for
higher-quality talent with the
ability to flag inappropriate content.
Simultaneously monitor language
Expand AI efforts to allow for higher-
quality talent with the ability to flag
inappropriate content.
How do CLs feel about FL quality today?
Expand AI efforts to allow for higher-quality talent with the ability to flag inappropriate content via Natural Language Processing (NLP)
Goals
Freelance talent badges no longer guarantee legitimate talent. With the introduction of AI and ability to further monitor content via Natural Language Processing (NPL), we will be able to monitor and flag suspicious activity.
Have a Project In Mind?
Let's Create Together!
Reach out through the form, and I'll respond within the next 24 hours.
