FastAPI

Marketplace Functionality in FastAPI

June 18, 2020 UPDATED ON: March 18, 2021

Welcome to Part 11 of Up and Running with FastAPI. If you missed part 10, you can find it here .

This series is focused on building a full-stack application with the FastAPI framework. The app allows users to post requests to have their residence cleaned, and other users can select a cleaning project for a given hourly rate.

Previously, we set up our database to gives users ownership over the cleaning resources they create. Now, we want to allow users to interact by giving them the ability to bid on jobs. We also want the owners to be able to approve and reject bids, just like a marketplace.

To do that, we’re going to need to update our database quite a bit.

Migrations

Before we make any changes, let’s roll back our migrations.

And now for our changes.

Our new user_offers_for_cleanings table connects a user who wants to offer their services and the cleaning they’re willing to work on. We also add a status column to indicate the state of the offer. If the owner of the cleaning has approved the cleaner for this job, the status will be “accepted”. If another cleaner gets the job, the status will be “rejected”. Until a decision has been made, the status will be “pending”.

Then we create a primary key composed of the cleaning id and the cleaner’s id. By using these two columns to create a primary key, we are ensuring that only unique combinations are allowed. So a user can only offer their services for a particular cleaning job once.

Many-to-Many relationships model situations where any number of items from the first table could be paired with any number of items from the second table. So how should we structure our endpoints? Does this live under the cleanings router or the users router? Does it get its own router?

We’ll get to all that in a minute. First, let’s tee up some models for our offers.

Offer Models

Go ahead and create a new file.

And then create a few new models.

Ooh. Lot’s of fun stuff happening here. We’re specifying that the status of the offer can either by pending, accepted, rejected, or cancelled. We’ll talk more about each state later on in the post. When an offer is created, we’re specifying that only the user_id and cleaning_id should be set. We’ll default the status to “pending” regardless. The status is also the only item we’re allowing users to update as well.

Note that the OfferInDB does not have the IDModelMixin, as we’re using a combination of the cleaning_id and user_id to construct the primary key in postgres. So we don’t actually ever have an ID for this model. Don’t worry, it won’t change anything.

In the OfferPublic model, we’re also specifying that we can attach the actual UserPublic or CleaningPublic models if we see fit.

Great. Now we can start writing some fancy new tests.

Setting Up the Offers Routes

Create a new test file.

Then add a test class to the file to check that our offers routes exist.

We have some standard REST endpoints, except we have two PUT routes. We’ll see why in a moment.

We run our tests and watch them fail. Getting these passing isn’t too much trouble.

Let’s make some dummy routes.

First, create the new file.

Then add a few routes that return None.

Now, the question here is how should we namespace our endpoints? We might think it’d be fine to just put them under /offers, but that’s probably not the best idea.

A better approach might look something like this:

Notice how we’re namespacing our offers_router under the prefix "/cleanings/{cleaning_id}/offers"? For every route in the offers_router, we’ll have the cleaning_id path parameter available representing the id of a cleaning resource. By doing so, we’ve made it easy to reuse our get_cleaning_by_id_from_path dependency we created in our previous post.

Run the tests again and they should pass.

Time for more tests.

Creating Offers

An offer is when a user wishes to provide their services for another user’s cleaning job. To test that this functionality works properly, we’ll need a user with a cleaning job and a few different users who will bid on the job.

Head into the conftest.py file and update it with a few additional fixtures, along with a helper method for creating users.

We’re extracting the logic required for fetching or creating a user from our database into its own helper and making 6 test users in total.

On top of that, we should create a helper fixture that constructs an authorized_client for any user we pass in. This will help us test authorized requests from multiple different users.

Now in our test_offers file we’ll be able to have users create offers for a given cleaning resource.

Let’s start with the TestCreateOffers class.

Five new tests here.

We’re ensuring that users can’t create offers for their own cleaning jobs, that unauthenticated user can’t create offers, and that users can’t create more than one offer for a given cleaning job. We also make sure the expected response is returned with properly structured requests and that we get the right error code when things are off.

To get those passing, open up the routes file again.

Ok - this makes sense. We check that a user is logged in and we check to make sure that user isn’t the owner of the cleaning resource that an offer is being created for. This’ll work, but whenever we start implementing permissions in the route like this, it’s a clear sign to use dependencies. Let’s refactor.

Much cleaner. We’re extracting our permissions checking into the api/dependencies/offers.py file and adding the check_offer_create_permissions function to our path decorator’s dependencies list. Readers who don’t recognize this pattern are encourage to read the previous post on owning resources in FastAPI.

Let’s go ahead and create the offers dependency file.

Then add this code:

There we go. Simple as pie. We’re leveraging our previously defined get_cleaning_by_id_from_path which does most of the hard work for us. We’ll be returning to this file a lot, so keep it open.

We still haven’t created our OffersRepository yet, so let’s do that next.

And in the file:

Now this works. Run the tests and they should all pass. However…this should again give us pause. Catching a unique violation error as a way to prevent duplicate entries is another permissions issue. When we see this, our thinking should go directly to dependencies.

We’ll refactor it in a minute.

First, let’s make sure we can fetch offers.

Getting Offers

Before we do anything else, we’re going to add a new fixture that’ll make our testing lives easier.

Add the following to tests/conftest.py:

Pretty self-explanatory. The test_cleaning_with_offers fixture creates a cleaning resource owned by test_user2 and then creates an offer from each user in the test_user_list.

Back to the test_offers.py file:

There’s a lot going on here, but it’s all connected. We’re testing five conditions:

1. We make sure that the owner of a cleaning job can successfully fetch a single offer made for that cleaning job.
2. On top of that, the creator of the offer should be able to fetch their own offer.
3. We ensure non-owners are forbidden from fetching an offer made for other users’ cleaning jobs.
4. We also make sure cleaning owners can fetch a list of all offers made for their own cleaning job.
5. Finally, non-owners are forbidden from fetching a list of offers made for a cleaning resource they don’t own.

Now let’s get them passing.

Dependency Fiesta

First off, recall that we’ll be fetching offers according to the id of the cleaning job and the username of the user who made the offer. So, let’s start by creating a dependency to fetch users by username from a path parameter.

And in the new file:

Tada! That’s an easy one. We already had our get_user_by_username method on our UsersRepository. All we needed to do was validate the username path parameter using the same code as our UsersCreate model and look up the user in the database.

Now update the api/dependencies/offers.py file like so:

That’s a lot of code.

First we have a get_offer_for_cleaning_from_user utility function that returns an OfferInDB object or a 404 response. It uses the yet-to-be-created method on OffersRepository which is also named get_offer_for_cleaning_from_user to look up that record in the database.

That utility function powers our get_offer_for_cleaning_from_user_by_path dependency that is used all over. It combines the get_user_by_username_from_path dependency we defined a second ago and the get_cleaning_by_id_from_path dependency we defined in our previous post. By doing so, it can pass all the required data to our utility function and look up the offer in the database.

A similar pattern is used to create the list_offers_for_cleaning_by_id_from_path dependency, which will be used to grab all existing offers for a given cleaning job.

We also update the check_offer_create_permissions with an additional check for uniqueness. Now we can remove that clunky try...except UniqueViolationError... block from our OffersRepository.

Last, but not least, we define permission checkers for listing offers and getting a single offer. In check_offer_list_permissions we ensure that only the cleaning owner can list offers for their own cleaning. In check_offer_get_permissions we forbid anyone except the cleaning owner and offer creator from getting a particular offer.

As tedious as this may seem, it moves permissions logic out of the routes and keeps them nice and tidy. Update the routes like so:

Look at that! The function body for both of our routes is a single line. Not bad.

All we’re missing is the database interface.

By removing all permission checking from our repository, this becomes simply an exercise in SQL.

Run the tests and they should all pass.

Accepting and Rejecting Offers

Now onto a slightly trickier part. To fulfill our marketplace requirements, cleaning owner should be able to accept and reject offers. Testing this properly is key to ensuring that our application is functioning the way we intend it to, so we’ll pay special attention to this next part.

Here’s what we want to accomplish:

1. The owner of a cleaning resource should be able to accept an offer, and no one else should be able to.
2. Only one offer should be accepted.
3. Once an offer is accepted, all others should be set to "rejected".

Add the following test case to our tests/test_offers.py file:

Hopefully by now these tests don’t look foreign. Feel free to add more if they don’t feel sufficient.

Regardless, we’ll get them passing in the same fashion as before. Start by adding a permission checker to the api/dependencies/offers.py file:

Nothing new here. We’re checking that the currently authenticated user must be the owner of the cleaning resource to accept it, and we’re ensuring that the current state of the offer is "pending".

Now let’s use it in the api/routes/offers.py file:

More of the same. This pattern has emerged in every one of our routes. Use the dependency injection system for supplying the database interface, gathering the correct resources, and checking permissions. Then modify database records and return the proper response.

In our OffersRepository:

Ah! Something new. In our accept_offer method, we’re managing a transaction through an async context block as specified in the encode databases documentation . The Postgres documentation explains transactions like so:

The essential point of a transaction is that it bundles multiple steps into a single, all-or-nothing operation. The intermediate states between the steps are not visible to other concurrent transactions, and if some failure occurs that prevents the transaction from completing, then none of the steps affect the database at all.

So, what’s the reason for using a transaction in our accept_offer method?

Well, we’re executing multiple queries that depend on each other. The crucial thing to understand about transactions is that either every database action succeeds, or they all fail. If there is an error in the ACCEPT_OFFER_QUERY or in the REJECT_ALL_OTHER_OFFERS_QUERY, then entire transaction block stops executing and all database changes will be rolled back.

When an owner accepts an offer for their cleaning job, we change the status of the offer to "accepted" and change all others to "rejected". We need all of that to happen or none of it. Any state in between would leave our database out of sync. By executing both operations inside a transaction, we ensure that we’re left with a state that’s manageable. We could definitely add additional error handling here, but for the time being, we’ll leave it as is.

Also notice how we don’t really need the offer_update parameter? Feel free to remove it here and in the route later on.

Run the tests and watch them all pass! Fantastic.

Rescinding and Cancelling Offers

Finally. We can see the finish line.

For this last part, we want two things. First, users who have made offers should to be able to rescind their own if it is still pending. Second, if the offer has already been accepted, we want users to be able to cancel their offer. Maybe they already accepted another offer, or they realize they just don’t have enough time.

We’ll start with cancelling.

This will look similar to our “accept” functionality, so let’s get to it.

Cancelling

Functionality allowing a user to cancel an offer will follow these guidelines:

1. Only the user who made the offer can cancel it
2. The offer must have already been accepted
3. All "rejected" offers for that cleaning job resource need to be set to "pending"

We’ll need to adequately test that.

To let users cancel their offer we’re going to need another new fixture. So add this to the tests/conftest.py file:

We’re creating a cleaning with offers from everyone in the test_user_list and having test_user2 accept the offer from test_user3.

Now, we add some tests.

Look familiar? That’s because it’s almost identical to accepting an offer. The biggest difference will be in the dependencies. Both in how we fetch the offer in question and how we specify permissions.

Open the api/dependencies/offers.py file and add two new dependencies.

We’re no longer fetching the offer using the username and id path parameter like we did with the get_offer_for_cleaning_from_user_by_path dependency. Instead we’re taking the currently authenticated user and combining the cleaning resource injected by the get_cleaning_by_id_from_path dependency to fetch the offer. Then we’re simply ensuring that the offer has already been accepted in the check_offer_cancel_permissions function.

And in the api/routes/offers.py file:

This route looks almost identical to the “accept” one. The biggest difference is that we don’t need the a username path parameter, since a currently authenticated user can only cancel their own offer.

The database interface isn’t much different either

In the OffersRepository:

Again, we’re using a transaction here to cancel the current offer and set all other offers to pending again. By this point we’re so close, we can smell it.

Run the tests and they should pass.

Rescinding

Now head into the test_offers.py file and add the last test class.

Here are the conditions we’re testing:

1. If a user submits an offer, they should be able to rescind it successfully if it is pending. It should no longer appear when listing all offers for a cleaning job.
2. If the offer is already accepted, raise a 400 exception. They should be forced to cancel it instead.
3. If the offer is already cancelled, they shouldn’t be able to cancel it again.
4. If the offer has been rejected, they shouldn’t be able to rescind it. We’ll keep the rejection as a record.

To get these passing, we’ll add one last dependency:

Same as before. We grab the offer in question and ensure that its status is pending. Otherwise we raise an exception.

Top it off by using it in the proper route:

More of the same, just with a DELETE action this time. All that’s left is to define the rescind_offer method.

And we polish off the OffersRepository with:

Run the tests again and they should all pass.

Wrapping Up and Resources

I’m exhausted, but happy. A functional marketplace has begun to emerge from our API and we have tested the entire thing. There’s more we could do here, but not without a proper rest.

In the next post, we’ll add an evaluation system for jobs that have been completed.

• Postgresql transaction docs
• Encode databases docs
• FastAPI path parameters docs
• FastAPI route decorator dependency docs
• FastAPI SQL databases docs
• Toptal: High-performing Apps with Python – blog post focused on building a todo app from scratch with FastAPI using the SQLAlchemy ORM.

Github Repo

All code up to this point can be found here:

• https://github.com/Jastor11/phresh-tutorial/tree/part-11-marketplace-functionality-in-fastapi

Special thanks to Valon Januzaj for correcting errors in the original code.

← Previous Post

User Owned Resources in FastAPI

June 12, 2020

Next Post →

Evaluations and SQL Aggregations in FastAPI

July 21, 2020