Working with email in development

Ensuring compatibility with mailer Sidekiq jobs

A Sidekiq job is enqueued whenever deliver_later is called on an ActionMailer. If a mailer argument needs to be added or removed, it is important to ensure both backward and forward compatibility. Adhere to the Sidekiq steps for changing the arguments for a worker.

The same applies to a new mailer method, or a new mailer. If you introduce either, follow the steps for adding new workers.

In the following example from NotificationService adding or removing an argument in this mailer’s definition may cause problems during deployment before all Rails and Sidekiq nodes have the updated code. 

mailer.unknown_sign_in_email(user, ip, time).deliver_later

Sent emails

To view rendered emails “sent” in your development instance, visit /rails/letter_opener.

S/MIME signed emails cannot be currently previewed with letter_opener.

Mailer previews

Rails provides a way to preview our mailer templates in HTML and plaintext using sample data.

The previews live in app/mailers/previews and can be viewed at /rails/mailers.

See the Rails guides for more information.

Incoming email

  1. Go to the GitLab installation directory.

  2. Find the incoming_email section in config/gitlab.yml, enable the feature and fill in the details for your specific IMAP server and email account:

    Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com: 

    incoming_email:
  enabled: true

  # The email address including the %{key} placeholder that will be replaced to reference the
  # item being replied to. This %{key} should be included in its entirety within the email
  # address and not replaced by another value.
  # For example: emailaddress+%{key}@gmail.com.
  # The placeholder must appear in the "user" part of the address (before the `@`). It can be omitted but some features,
  # including Service Desk, may not work properly.
  address: "gitlab-incoming+%{key}@gmail.com"

  # Email account username
  # With third party providers, this is usually the full email address.
  # With self-hosted email servers, this is usually the user part of the email address.
  user: "gitlab-incoming@gmail.com"
  # Email account password
  password: "[REDACTED]"

  # IMAP server host
  host: "imap.gmail.com"
  # IMAP server port
  port: 993
  # Whether the IMAP server uses SSL
  ssl: true
  # Whether the IMAP server uses StartTLS
  start_tls: false

  # The mailbox where incoming mail will end up. Usually "inbox".
  mailbox: "inbox"
  # The IDLE command timeout.
  idle_timeout: 60

  # Whether to expunge (permanently remove) messages from the mailbox when they are marked as deleted after delivery
  expunge_deleted: false

    As mentioned, the part after + is ignored, and this message is sent to the mailbox for gitlab-incoming@gmail.com.

  3. Read the MailRoom Gem updates section for more details before you proceed to make sure you have the right version of MailRoom installed. In summary, you need to update the gitlab-mail_room version in the Gemfile to the latest gitlab-mail_room temporarily and run bundle install. Do not commit this change as it’s a temporary workaround.

  4. Run this command in the GitLab root directory to launch mail_room: 

    bundle exec mail_room -q -c config/mail_room.yml

  5. Verify that everything is configured correctly: 

    bundle exec rake gitlab:incoming_email:check RAILS_ENV=development

  6. Reply by email should now be working.

Email namespace

As of GitLab 11.7, we support a new format for email handler addresses. This was done to support catch-all mailboxes.

If you need to implement a feature which requires a new email handler, follow these rules for the format of the email key:

  • Actions are always at the end, separated by -. For example -issue or -merge-request
  • If your feature is related to a project, the key begins with the project identifiers (project path slug and project ID), separated by -. For example, gitlab-org-gitlab-foss-20
  • Additional information, such as an author’s token, can be added between the project identifiers and the action, separated by -. For example, gitlab-org-gitlab-foss-20-Author_Token12345678-issue
  • You register your handlers in lib/gitlab/email/handler.rb

Examples of valid email keys:

  • gitlab-org-gitlab-foss-20-Author_Token12345678-issue (create a new issue)
  • gitlab-org-gitlab-foss-20-Author_Token12345678-merge-request (create a new merge request)
  • 1234567890abcdef1234567890abcdef-unsubscribe (unsubscribe from a conversation)
  • 1234567890abcdef1234567890abcdef (reply to a conversation)

The action -issue- is used in GitLab as the handler for the Service Desk feature.

Legacy format

Although we continue to support the older legacy format, no new features should use a legacy format. These are the only valid legacy formats for an email handler:

  • path/to/project+namespace
  • path/to/project+namespace+action
  • namespace
  • namespace+action

In GitLab, the handler for the Service Desk feature is path/to/project.

MailRoom Gem updates

We use gitlab-mail_room, a fork of MailRoom, to ensure that we can make updates quickly to the gem if necessary. We try to upstream changes as soon as possible and keep the two projects in sync.

Updating the gitlab-mail_room dependency in Gemfile is deprecated at the moment in favor of updating the version in Omnibus (see example merge request) and Helm Chart configuration (see example merge request).

Rationale

This was done because to avoid thread deadlocks, MailRoom needs an updated version of the net-imap gem. However, this version of the net-imap cannot be installed by an unprivileged user due to an error installing the digest gem. This bug in the Ruby interpreter was fixed in Ruby 3.0.2.

Updating the gem directly in the GitLab Rails Gemfile caused a production incident since Kubernetes pods do not run as privileged users.

Note that Omnibus GitLab runs /opt/gitlab/embedded/bin/mail_room instead of bundle exec rake to avoid loading the older version. With a Kubernetes install, the MailRoom version has always been explicitly set in the Helm Chart configuration rather than the Gemfile.

Preserving backwards compatibility

Removing the Gemfile would break incoming email processing for source installs. For now, source installs are advised to upgrade manually to the version specified in Omnibus and run bin/mail_room directly as done with Omnibus.

We can reconsider this deprecation once we upgrade to Ruby 3.0.

Return to Development documentation