What's new in Black 22.10.0

On October 6th, 2022, we released Black 22.10.0. It’s a smaller release by the size of the changelog, but hey that means I don’t have to write as much!

This is meant to be an accompanying blog post for 22.10.0. I won’t discuss every change, just the ones worth highlighting. Instead, I’ll go into more detail, providing context and explanations (and probably some behind of scenes stuff too – this isn’t official anyway). I know this is a bit late, but I hope this is still insightful.

If you’d like the full changelog for 22.10.0, please refer to the link above.

Packaging

Goodbye Python 3.6 (and hello Hatchling!)

It’s official: you can no longer run Black on Python 3.6 starting with 22.10.0.¹

This is actually the first time Black has ever dropped runtime support since 3.6+ was required from the start.²

We dropped Python 3.6 because we wanted to switch build backends from Setuptools to Hatchling which only supports 3.7+. Setuptools has served us well as a simple and reliable backend, but we wanted to modernize. While Setuptools does finally now support most packaging standards (PEP 517, PEP 621, PEP 660 and more) our needs are a bit unique.

We use mypyc to compile Black for a significant performance boost. Before Hatchling we hand-rolled our own integration using mypyc’s basic Setuptools build API. It worked well enough, but it meant we would be stuck with setup.py forever until a Setuptools plugin for mypyc is available. (one does not exist as of writing)

All of the reasons behind the switch can be found in the PR, but in essence, Hatchling allowed us to move to static metadata in pyproject.toml while providing us with a nicer mypyc integration thanks to its hatch-mypyc plugin.

Switching to Hatchling hasn’t been as simple as I would’ve wished: we had to track down this confusing linker error that only appears when using build isolation (among other situations) and now our macOS builds are producing mislabelled wheels, ugh. It happens.

Compiled wheels for 3.11

We couldn’t build mypyc wheels for CPython 3.11 since we had been using an older manylinux docker image to avoid the linker error mentioned earlier. It was old enough that 3.11 wasn’t pre-installed!

Once the linker error was addressed by @zsol (thank you, I had no chance of figuring it out), it was pretty straightforward to configure cibuildwheel to build CPython 3.11 wheels. Just a CIBW upgrade, a type error fix, and a workaround so that aiohttp doesn’t fail to install on 3.11.

Code style

Failing to put fmt: on at the same indent level as fmt: off no longer crashes

Previously Black produced invalid code when # fmt: on is placed on a different indent level than the # fmt: off it’s paired with. Black failed similarly when # fmt: on just didn’t exist.

def x():
    # fmt: off
    return

def y():
    return

$ black issue-569.py
error: cannot format issue-569.py: INTERNAL ERROR: Black produced code that is not equivalent to the source.  Please report a bug on https://github.com/psf/black/issues.  This diff might be helpful: /tmp/blk_wv8p1p5k.log

Oh no! 💥 💔 💥
1 file failed to reformat.

If you disable the safety checks with --fast, you’ll quickly realize that Black badly messed up the indentation.

def x():
    # fmt: off
    return

    def y():
        return

This annoying bug was often hit. If you look at the original issue’s timeline, you’ll notice that this bug has been reported many, many times over. Yet, this bug was left unfixed for almost three years until @yilei opened GH-3281 to finally fix it.

Black still requires # fmt: off and # fmt: on to be used at the same indent level, but now, the code at or above the initial # fmt: off’s block level will be left untouched, when # fmt: on is used on a different level or there is no # fmt: on.

In other words, you can now use a single # fmt: off to turn off formatting by block level.

def ignore_this_function(condition):
    # fmt: off
    if condition:
        return 'hello, world'


def format_this_function():
    def except_this_block_level():
        # fmt: off
        return 'hello, world'

    return except_this_block_level ()

$ black file.py --diff --color
--- file.py	2022-10-15 18:30:53.887339 +0000
+++ file.py	2022-10-15 18:30:54.780816 +0000
@@ -7,6 +7,6 @@
 def format_this_function():
     def except_this_block_level():
         # fmt: off
         return 'hello, world'

-    return except_this_block_level ()
+    return except_this_block_level()

Anyway, I’m so happy this major bug has finally been fixed. Thank you so much @yilei!

Preview style: fixed a crash related to string keys in dictionaries

Parentheses are added around implicitly concatenated strings in various situations under the preview style since 22.8.0. Unfortunately, this introduced a bug where long string keys would be wrapped in extra parentheses along with the value.

v = {
    ('a_very_long_str_a_very_long_str_a_very_long_str_a_very_long_str_'
     'a_very_long_str_a_very_long_str_a_very_long_str_'): {
        'key':
            'value',
    },
}

 v = {
-    ('a_very_long_str_a_very_long_str_a_very_long_str_a_very_long_str_'
-     'a_very_long_str_a_very_long_str_a_very_long_str_'): {
-        'key':
-            'value',
-    },
+    (
+        "a_very_long_str_a_very_long_str_a_very_long_str_a_very_long_str_a_very_long_str_a_very_long_str_a_very_long_str_": {
+            "key": "value",
+        }
+    ),
 }

Now Black no longer adds these invalid parentheses, although it just merges the strings together violating the line length which doesn’t seem right…

GitHub Action improvements

The GitHub Action can now format Jupyter Notebooks if you set jupyter: true:

- uses: psf/black@stable
  with:
    src: "./src"
    jupyter: true
    version: "22.10.0"

Under the hood, it will install Black with the jupyter extra enabling Black’s optional support for .ipynb files.

Additionally, the version key now supports PEP 440 version specifiers. This is a great QOL improvement because if you want to match versions covered by Black’s stability policy, you can use the compatible release operator (~=) to do that now:

- uses: psf/black@stable
  with:
    src: "./src"
    version: "~= 22.0"

By restricting to 2022 releases, the latest version possible will be used without sudden changes (breaking your CI) since releases within the same year are guaranteed to not change the stable style.

This was a backwards-compatible change, you can still specify a single version if you’d like.

-x / --skip-source-first-line

Python has a flag (-x) to skip the first line of files given. I didn’t know this existed, but it turns out it’s commonly used with Python WASM pages. They shove the HTML code in a single line at the start as a non-standard “shebang” (since browsers generally require HTML files to start with <) so it can run in the browser and locally.

For example, this browser pong game uses pygame via pygbag (a WASM version of pygame). The start of the HTML file looks like this:

<html><head><meta charset="utf-8"></head><script src="https://pygame-web.github.io/archives/0.3/pythons.js" type=module id="__main__" data-src="gui" async defer>#<!--
import sys
import os
import asyncio

Since all the HTML code is at the top in a single line, you can run it as any other Python script by simply passing -x.

wget https://pygame-web.github.io/showroom/pygame-scripts/org.pygame.touchpong.html
# assuming pygame is installed
python -x org.pygame.touchpong.html

I was hesitant to add a similar option to Black, but it was a convincing use-case and the issue got a surprising amount of upvotes.

This option is also available for Blackd.

… and that’s all. Thanks for reading! ❀