Grokking Django
Vote

0% completed

​

ManyToManyField, OneToOneField and Efficient Queries

Many-to-Many: ManyToManyField

One-to-One: OneToOneField

Loading Related Rows Efficiently

Choosing the Right Field

Key Takeaways

One column is enough to link an article to its author, because each article has exactly one author. The blog from the previous lesson now has that link: Article.author is a ForeignKey to Author, Comment.article is a ForeignKey to Article, and the database holds three articles by Ana Lopez and Ben Carter. Two kinds of link are still missing. Tags need a link with many partners on both sides, and user profiles need a link with exactly one partner on each side. After adding both, the lesson shows how to load related rows without running one query per row.

Many-to-Many: ManyToManyField

An article can have several tags, such as "django" and "beginners", and a tag labels many articles. Neither table can hold this link in one column, because each side has many partners. The answer is a third table, called a join table, in which each row pairs one article with one tag. The ManyToManyField creates that table for you:

# blog/models.py class Tag(models.Model): name = models.CharField(max_length=50, unique=True) def __str__(self): return self.name class Article(models.Model): # ...the fields shown earlier... tags = models.ManyToManyField(Tag, blank=True)

The other fields of Article stay as the previous lesson left them. Put Tag above Article in the file. The blank=True option lets forms accept an article with no tags. The migration, 0006_tag_article_tags.py, adds no column to blog_article. It creates the join table instead:

CREATE TABLE "blog_article_tags" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "article_id" bigint NOT NULL REFERENCES "blog_article" ("id") DEFERRABLE INITIALLY DEFERRED, "tag_id" bigint NOT NULL REFERENCES "blog_tag" ("id") DEFERRABLE INITIALLY DEFERRED); CREATE UNIQUE INDEX "blog_article_tags_article_id_tag_id_b78a22e9_uniq" ON "blog_article_tags" ("article_id", "tag_id");

The unique index means one article cannot carry the same tag twice. You link and unlink rows with add(), remove() and set(), which insert and delete rows of the join table. The example uses the three articles from the previous lesson. If you start a new shell session, load the first one again, because the variable first from that lesson no longer exists:

from blog.models import Article first = Article.objects.get(title="Getting Started with Django")

Then create three tags and link them:

from blog.models import Article, Tag second = Article.objects.get(title="Django Models in Practice") third = Article.objects.get(title="Understanding Querysets") django_tag = Tag.objects.create(name="django") beginners = Tag.objects.create(name="beginners") orm = Tag.objects.create(name="orm") first.tags.add(django_tag, beginners) second.tags.add(django_tag, orm) third.tags.add(orm) first.tags.add(django_tag) # already linked, so nothing changes print(first.tags.all()) print(orm.article_set.all()) print(Article.objects.filter(tags__name="orm"))
<QuerySet [<Tag: django>, <Tag: beginners>]> <QuerySet [<Article: Django Models in Practice>, <Article: Understanding Querysets>]> <QuerySet [<Article: Django Models in Practice>, <Article: Understanding Querysets>]>

The reverse side follows the same naming rule as ana.article_set in the previous lesson, so a tag reaches its articles through article_set. Filtering by tags__name makes Django join all three tables. The call first.tags.remove(beginners) deletes one pairing, and first.tags.set([django_tag, beginners]) replaces the article's tags with exactly that list.

Both objects must be saved before you link them, because a join row needs two ids. Adding a tag to an article that has no id yet fails with ValueError: "<Article: x>" needs to have a value for field "id" before this many-to-many relationship can be used.

One-to-One: OneToOneField

The project's login accounts are rows of the User model that you created in the accounts app in Chapter 1, and the chapter on authentication explains it. Suppose each user should also have a short biography and a birth date, shown next to their articles. You could add those fields to User, but they belong to the blog, not to logging in. A common choice is to keep them in the blog app, in a profile model with exactly one row per user, linked by a OneToOneField:

# blog/models.py from django.conf import settings class Profile(models.Model): user = models.OneToOneField(settings.AUTH_USER_MODEL, on_delete=models.CASCADE) bio = models.TextField(blank=True) birth_date = models.DateField(null=True, blank=True) def __str__(self): return f"Profile of {self.user.username}"

The settings.AUTH_USER_MODEL setting names the project's user model, which in this project is the string "accounts.User". Models always refer to the user model through this setting, not by importing the class, so that blog does not depend on where the user model lives. The null=True and blank=True on birth_date make the date optional, in the database and in forms.

A OneToOneField is a foreign key with a uniqueness rule. The SQL for 0007_profile.py shows both parts in one column:

"user_id" bigint NOT NULL UNIQUE REFERENCES "accounts_user" ("id") DEFERRABLE INITIALLY DEFERRED

Because each user has at most one profile, the reverse side is a single object, not a manager, and its name is the model name in lowercase, profile:

import datetime from django.contrib.auth import get_user_model from blog.models import Profile User = get_user_model() user = User.objects.create_user(username="ana", password="s3cret-pass-123") Profile.objects.create(user=user, bio="Writes about Django.", birth_date=datetime.date(1990, 5, 20)) print(user.profile.bio) print(user.profile.user.username)
Writes about Django. ana

The get_user_model() function returns the class that AUTH_USER_MODEL names, here accounts.models.User, so code outside the accounts app does not import it by name. The create_user() method stores the password as a hash rather than as plain text. When a user has no profile, reading user.profile raises User.profile.RelatedObjectDoesNotExist: User has no profile., and hasattr(user, "profile") returns False. A second profile for the same user is refused by the database: IntegrityError: UNIQUE constraint failed: blog_profile.user_id.

A profile picture would be one more field, an ImageField. That field needs the Pillow library (pip install Pillow), and uploaded files need settings that this chapter does not cover.

The blog's tables with their foreign keys, the unique key of the profile, and the join table between articles and tags.
The blog's tables with their foreign keys, the unique key of the profile, and the join table between articles and tags.

The diagram puts all three relationships side by side. Notice where each link is stored: a ForeignKey is a column on the "many" side, a OneToOneField is the same kind of column with UNIQUE added, and a ManyToManyField lives in a table of its own.

Following a link is convenient, but each first read of article.author runs its own query. A loop over articles that prints each author's name runs one query for the articles and then one more for every article:

from django.db import connection from django.test.utils import CaptureQueriesContext with CaptureQueriesContext(connection) as ctx: for article in Article.objects.all(): print(article.title, "by", article.author.name) print("queries:", len(ctx.captured_queries))
Getting Started with Django by Ana Lopez Django Models in Practice by Ana Lopez Understanding Querysets by Ben Carter queries: 4

Three articles cost four queries, and a page of a hundred articles would cost 101. This pattern is known as the N+1 query problem. The fix is to tell Django in advance which related rows you will need. For a ForeignKey or a OneToOneField, select_related() adds a JOIN to the first query, so the authors arrive with the articles. Counted inside the same CaptureQueriesContext block, this loop prints the same three lines and runs one query:

for article in Article.objects.select_related("author"): print(article.title, "by", article.author.name)
queries: 1 SELECT ..., "blog_author"."id", "blog_author"."name", "blog_author"."email" FROM "blog_article" INNER JOIN "blog_author" ON ("blog_article"."author_id" = "blog_author"."id")

A join does not work well for a many-to-many link or a reverse link, because one article can match many tags and the joined rows would repeat. For those links, prefetch_related() runs one extra query for all the related rows and matches them to the articles in Python. Counted the same way:

for article in Article.objects.prefetch_related("tags"): print(article.title, [tag.name for tag in article.tags.all()])
Getting Started with Django ['django', 'beginners'] Django Models in Practice ['django', 'orm'] Understanding Querysets ['orm'] queries: 2

The second query fetches the tags of all three articles at once with WHERE "blog_article_tags"."article_id" IN (1, 2, 3). Without prefetch_related(), the same loop runs four queries.

Current Django can also do this batching for you when you do not know in advance which links a loop will read. With Article.objects.fetch_mode(models.FETCH_PEERS), where models is django.db.models, the first read of article.author loads the authors of all the articles fetched together, so the author loop above runs two queries instead of four.

The same loop over three articles costs four queries without select_related and one query with it.
The same loop over three articles costs four queries without select_related and one query with it.

Compare the two columns of the picture: the number of queries on the left grows with the number of articles, while on the right it stays at one. Use select_related() and prefetch_related() whenever a loop or a template reads a related object for every row.

Choosing the Right Field

The dataFieldWhere it goesBlog example
Each row has one partner, and the partner has manyForeignKeythe "many" sideArticle.author, Comment.article
Each row has many partners on both sidesManyToManyFieldeither side (one side only)Article.tags
Each row has at most one partner on both sidesOneToOneFieldthe side that extends the otherProfile.user

When you are unsure, ask the question from both sides. "Can one author have many articles?" Yes. "Can one article have many authors?" In this blog, no, so the link is a ForeignKey on Article. If the answer to the second question were yes, the link would be a ManyToManyField.

Key Takeaways

  • A ManyToManyField creates a join table, such as blog_article_tags, and you link rows with add(), remove() and set().
  • A OneToOneField is a unique foreign key, and its reverse side is a single object. Refer to the user model through settings.AUTH_USER_MODEL.
  • A loop that reads a related object for every row runs one query per row. select_related() fixes it for foreign keys and one-to-one links, and prefetch_related() fixes it for many-to-many and reverse links.

Reading Progress

0%


Vote for new content

On This Page

Many-to-Many: ManyToManyField

One-to-One: OneToOneField

Loading Related Rows Efficiently

Choosing the Right Field

Key Takeaways