Auto-capturing multi-statement closures

[nunomaduro]

As you may already know, PHP 7.4 has introduced one-liner arrow functions (aka short closures). Now, this pull request adds the possibility of those arrow functions to be multi-line. Let's see an example:

$users = [/** */];
$guestsIds = [/** */];
$repository = /** */;

$guests = array_filter($users, fn ($user) {
    $guest = $repository->findByUserId($user->id);

    return $guest !== null && in_array($guest->id, $guestsIds);
});

This pull request should be targeting PHP 8.1, and it will be followed by an RFC in the next couple of days - but in short, the advantages are:

-$values = array_filter($values, function ($value) use ($first, $second, $third, $four) {
+$values = array_filter($values, fn ($value) {

• Multi-line arrow functions don't require the use keyword to be able to access data from the outer scope.
• Also, in some cases, fn (/** */) { is just shorter and simpler than function (/** */) use (/** */) {.

It is my first contribution to the core of PHP, so let me know if something in this pull request needs to be better. 👍🏻

[mvorisek]

add one more static test and check with reflection that $this is really not bound

[nunomaduro]

@mvorisek Gonna wait for the RFC decision before updating that.

[nikic]

This will need a more sophisticated capture analysis to ensure that

$x = 1;
$fn = fn() {
    $x = 2;
    return $x;
};

does not perform an unnecessary binding of $x from the outer scope. You should be able to test capture behavior using opcache dumps.

[nikic]

I wonder, why fn() => {} rather than just fn() {}?

[ConsoleTVs]

@nikic I feel the => looks better and just follows the consistency of other languages like Typescript / Javascript / Dart, etc.

[nikic]

While it may be consistent with other languages, it is more important to be internally consistent within PHP. Especially when I view this in conjunction with #6221, which introduces => expr for method bodies, it seems like we should be interpreting => expr; as a shorthand for { return expr; }. In this context => { stmt; } does not make sense syntactically.

Of course, there is also an alternative view here: If we add general support for block expressions, then the syntax => { stmt; } would make sense. But that would have larger implications, in that it should also be usable outside the context of arrow functions. You will want to read up on the discussions we had on this when considering block expressions for match.

[ConsoleTVs]

Totally agree on your point here. I was not aware of #6221, but it does in fact make sense. I would consider your second point as some sort of block expression like Rust have. Could you point me to the discussion regarding block expressions? It peaked my curiosity ;)

[nikic]

It's a bit spread out, https://externals.io/message/109941#109947, #5403, #5448 seem relevant, there's probably more in the other match discussion threads.

[nunomaduro]

@nikic I will make sure read those discussions before making an RFC. It may take a while.

[donmbelembe]

I think '=>' is not necessary since PHP has the 'fn' keyword

[nunomaduro]

Addressed here: b82fe0c.

[dragoonis]

Thank you, @nunomaduro

[HallofFamer]

This looks wonderful, hope you get it rolling, make an RFC, and get it into voting before the PHP 8.1 release.

[nunomaduro]

@nikic I've just updated the pull request so it makes more sense together with Short Functions. This way, only single line return expressions would have the => symbol:

$values = array_filter($values, fn ($value) { // no more `=>` symbol.

[nunomaduro]

cc @Crell

[Crell]

I don't really have strong feelings on this, frankly. The use clause is annoying, but for multi-line closures the longer function keyword doesn't bother me. As Nikita notes, block expressions are a whole other ball of complicated wax with very deep implications. Doing that as a one-off is a no-go. It would only make sense as a language-wide well-thought-through change to the language semantics.

[BenMorel]

Thanks a lot for this PR, @nunomaduro! I really hope that this proposal will move forward and be accepted. I just posted another message on @internals emphasizing how this would be useful to me.

Side question: would it be possible to drop the fn() => entirely, and create a closure when braces are encoutered outside of a function/class/interface/trait/..., or in other words, whenever an expression is expected?

Like this:

$connection->transactional({
    // do stuff
});

Right now it's a syntax error, and so is this:

$a = {};

So I don't think this would conflict with any current syntax?

[mvorisek]

@BenMorel how would it accept params? --> probably no :)

[BenMorel]

@mvorisek It wouldn't. This would be an alternate syntax that creates a closure with no parameters and inheriting the outside scope.

[tuqqu]

@BenMorel yet another alternate syntax just to save a few keystrokes and to add mental overhead when reading code. Not a fan tbh.

[stancl]

I think that {} is fine. It currently doesn't represent anything in PHP.

And if you had ($a, $b) => {}, there wouldn't be any issues either. Just like fn() was added to make recognizing arrow functions easier, here you have the {} which means that it couldn't possibly be anything else than an arrow function.

[Crell]

{} means all kinds of things in PHP today. It sets off a block of statements, which could be a function body but could also be the body of a for loop, or a switch statement, or a match expression, etc.

Alternate brainstorm: {{ }} to indicate "a block of statements that together form a single expression, with auto-capture". It's not used anywhere currently, but it would then allow you to bundle up statements anywhere you want an expression. That would work for multi-line short-lambdas as well as multi-line match arms (the other place where "multiline expressions" have come up).

So you'd get:

fn($x, $y) => {{ 
blah;
blah;
return blah;
}};

match($beep) {
  'beep' => {{
    blah;
    blah;
    return blah;
  }},
};

That would, in combination with short functions, technically make this feasible:

function foo($x, $y) => {{
  blah;
  blah;
  return blah;
}};

Which is somewhat weird but I don't think would hurt anything.

[nunomaduro]

@stancl We are preparing an RFC that should land this week or next week.

[nunomaduro]

RFC under discussion on PHP Mailing Lists > php.internals: https://wiki.php.net/rfc/auto-capture-closure.

[derickr]

When are you going to put this up for a vote?

[Crell]

@nunomaduro wanted to try and find time to investigate some memory optimizations, but hasn't been able to yet. It will be going to a vote as soon as he tells me he's either done so or isn't going to have time to do so. :-)

[mvorisek]

Add also these tests:

$a = 2;
$fn = fn() {
    $a++;
    var_dump($a);
};
$a++;
$a++;
var_dump($a); // should output 4
$fn();        // should output 3
var_dump($a); // should output 5

and

$a = 2;
$fn = fn($n) {
    ${$n}++;
    var_dump(${$n});
};
$fn(); // should output 3

[bobmagicii]

may i ask why it was up for vote, had a bunch... but then got reset? was the publishing for voting originally in error?

[mvorisek]

I belive because:

And I am not a fan of autocapturing as well.

[bobmagicii]

nah, the daily digest the mailing list just sent explained why.

[cmb69]

@nunomaduro, @Crell, are you planning to pursue the RFC for PHP 8.2?

[Crell]

I think that mainly depends on if @nunomaduro is able to do the memory optimizations discussed previously. If yes, then probably.

[nunomaduro]

@cmb69 I don't plan tackle those memory optimisations or make progress on this pull request. Of course, other developer is free to keep working with @Crell on this RFC / pull request.

[cmb69]

Thanks for the swift reply, @Crell and @nunomaduro! AIUI, it's not really necessary to have memory optimizations (at least for now). Quoting from a relevant mail by @nikic:

Here, there is no need to import $tmp from the outer scope, but a naive
implementation (which I assume you're using) will still try to capture it.
Now, this has two effects:

a) Performance, as we're trying to capture unnecessary variables. This may
be negligible, but it would be good to at least quantify. I would rather
there not be recommendations along the line of "you should use function()
for performance-critical code, because it's faster than fn()".

b) Subtle side-effects, visible in debugging functionality, or through
destructor effects (the fact that a variable is captured may be
observable). I think it nothing else, the RFC should at least make clear
that this behavior is explicitly unspecified, and a future implementation
may no longer capture variables where any path from entry to read passes a
write.

So what should be done is measuring/comparing the performance, and to add relevant clarification to the RFC. IOW, no implementation enhancement are strictly necessary.

[HallofFamer]

I hope this one will make it to PHP 8.2. As we know, the requirement for manual capturing of variables with annoying use statement is one of the worst features from PHP, there aint another language that forces you to do this. It is enough a big problem that I dont use anonymous functions/closures beyond single line statement(thanks to PHP 7.4 for the short closure), or simple multi-line statements that do not need to capture any variables(yes I dont write use statement for anonymous functions, its a blasphemy to me). However, its quite reasonable that we will need multi-line(at least 3-5 lines are rather common) statement in closures that will need to capture 1-2 outer variables, so adding this to PHP will be helpful.

I dont understand why some delusional people were against this saying its easy to abuse. Come on, every other mainstream language that supports closures work this way, what makes one think its prone to being abused in PHP? Most of these people have large chunk of procedural code with hundreds of variables who worry about accidentally capturing variables they dont need, then I am sorry its their problem for writing spaghetti code(and the easy solution to their problem is, just to use traditional anonymous functions, its not going to disappear anyway). Auto-capturing actually helps them avoid writing such long procedural spaghetti code, its a benefit rather than disadvantage to me.

Edit: lol why the dislikes? I cant imagine any sane developers who actually enjoy PHP's tedious use statement for manual capturing. Still, no one can provide me with an example that another mainstream language requires such use statement like PHP does currently(apparently this so-called scope-leak is never an issue for developers using other languages), its an abomination that needs to go away.

[iluuu1994]

@arnaud-lb is working on a new implementation with live variable analysis that only captures variables that aren't immediately overwritten (which was the main concern with this implementation). #8330 @nunomaduro Is it ok with you if I close this PR?

[nunomaduro]

@iluuu1994 Done. ✅