perl5260delta - what is new for perl v5.26.0



NAME

perl5260delta - what is new for perl v5.26.0


DESCRIPTION

This document describes the differences between the 5.24.0 release and the 5.26.0 release.


Notice

This release includes three updates with widespread effects:


Core Enhancements

Lexical subroutines are no longer experimental

Using the lexical_subs feature introduced in v5.18 no longer emits a warning. Existing code that disables the experimental::lexical_subs warning category that the feature previously used will continue to work. The lexical_subs feature has no effect; all Perl code can use lexical subroutines, regardless of what feature declarations are in scope.

Indented Here-documents

This adds a new modifier "~" to here-docs that tells the parser that it should look for /^\s*$DELIM\n/ as the closing delimiter.

These syntaxes are all supported:

    <<~EOF;
    <<~\EOF;
    <<~'EOF';
    <<~"EOF";
    <<~`EOF`;
    <<~ 'EOF';
    <<~ "EOF";
    <<~ `EOF`;

The "~" modifier will strip, from each line in the here-doc, the same whitespace that appears before the delimiter.

Newlines will be copied as-is, and lines that don't include the proper beginning whitespace will cause perl to croak.

For example:

    if (1) {
      print <<~EOF;
        Hello there
        EOF
    }

prints ``Hello there\n'' with no leading whitespace.

New regular expression modifier /xx

Specifying two "x" characters to modify a regular expression pattern does everything that a single one does, but additionally TAB and SPACE characters within a bracketed character class are generally ignored and can be added to improve readability, like /[ ^ A-Z d-f p-x ]/xx. Details are at /x and /xx in the perlre manpage.

@{^CAPTURE}, %{^CAPTURE}, and %{^CAPTURE_ALL}

@{^CAPTURE} exposes the capture buffers of the last match as an array. So $1 is ${^CAPTURE}[0]. This is a more efficient equivalent to code like substr($matched_string,$-[0],$+[0]-$-[0]), and you don't have to keep track of the $matched_string either. This variable has no single character equivalent. Note that, like the other regex magic variables, the contents of this variable is dynamic; if you wish to store it beyond the lifetime of the match you must copy it to another array.

%{^CAPTURE} is equivalent to %+ (i.e., named captures). Other than being more self-documenting there is no difference between the two forms.

%{^CAPTURE_ALL} is equivalent to %- (i.e., all named captures). Other than being more self-documenting there is no difference between the two forms.

Declaring a reference to a variable

As an experimental feature, Perl now allows the referencing operator to come after my(), state(), our(), or local(). This syntax must be enabled with use feature 'declared_refs'. It is experimental, and will warn by default unless no warnings 'experimental::refaliasing' is in effect. It is intended mainly for use in assignments to references. For example:

    use experimental 'refaliasing', 'declared_refs';
    my \$a = \$b;

See Assigning to References in the perlref manpage for more details.

Unicode 9.0 is now supported

A list of changes is at http://www.unicode.org/versions/Unicode9.0.0/. Modules that are shipped with core Perl but not maintained by p5p do not necessarily support Unicode 9.0. the Unicode::Normalize manpage does work on 9.0.

Use of \p{script} uses the improved Script_Extensions property

Unicode 6.0 introduced an improved form of the Script (sc) property, and called it Script_Extensions (scx). Perl now uses this improved version when a property is specified as just \p{script}. This should make programs more accurate when determining if a character is used in a given script, but there is a slight chance of breakage for programs that very specifically needed the old behavior. The meaning of compound forms, like \p{sc=script} are unchanged. See Scripts in the perlunicode manpage.

Perl can now do default collation in UTF-8 locales on platforms that support it

Some platforms natively do a reasonable job of collating and sorting in UTF-8 locales. Perl now works with those. For portability and full control, the Unicode::Collate manpage is still recommended, but now you may not need to do anything special to get good-enough results, depending on your application. See Category LC_COLLATE: Collation: Text Comparisons and Sorting in the perllocale manpage.

Better locale collation of strings containing embedded NUL characters

In locales that have multi-level character weights, NULs are now ignored at the higher priority ones. There are still some gotchas in some strings, though. See Collation of strings containing embedded NUL characters in the perllocale manpage.

CORE subroutines for hash and array functions callable via reference

The hash and array functions in the CORE namespace (keys, each, values, push, pop, shift, unshift and splice) can now be called with ampersand syntax (&CORE::keys(\%hash) and via reference (my $k = \&CORE::keys; $k->(\%hash)). Previously they could only be used when inlined.

New Hash Function For 64-bit Builds

We have switched to a hybrid hash function to better balance performance for short and long keys.

For short keys, 16 bytes and under, we use an optimised variant of One At A Time Hard, and for longer keys we use Siphash 1-3. For very long keys this is a big improvement in performance. For shorter keys there is a modest improvement.


Security

Removal of the current directory (".") from @INC

The perl binary includes a default set of paths in @INC. Historically it has also included the current directory (".") as the final entry, unless run with taint mode enabled (perl -T). While convenient, this has security implications: for example, where a script attempts to load an optional module when its current directory is untrusted (such as /tmp), it could load and execute code from under that directory.

Starting with v5.26, "." is always removed by default, not just under tainting. This has major implications for installing modules and executing scripts.

The following new features have been added to help ameliorate these issues.

Here are some things script and module authors may need to do to make their software work in the new regime.

Escaped colons and relative paths in PATH

On Unix systems, Perl treats any relative paths in the PATH environment variable as tainted when starting a new process. Previously, it was allowing a backslash to escape a colon (unlike the OS), consequently allowing relative paths to be considered safe if the PATH was set to something like /\:.. The check has been fixed to treat "." as tainted in that example.

New -Di switch is now required for PerlIO debugging output

This is used for debugging of code within PerlIO to avoid recursive calls. Previously this output would be sent to the file specified by the PERLIO_DEBUG environment variable if perl wasn't running setuid and the -T or -t switches hadn't been parsed yet.

If perl performed output at a point where it hadn't yet parsed its switches this could result in perl creating or overwriting the file named by PERLIO_DEBUG even when the -T switch had been supplied.

Perl now requires the -Di switch to be present before it will produce PerlIO debugging output. By default this is written to stderr, but can optionally be redirected to a file by setting the PERLIO_DEBUG environment variable.

If perl is running setuid or the -T switch was supplied, PERLIO_DEBUG is ignored and the debugging output is sent to stderr as for any other -D switch.


Incompatible Changes

Unescaped literal "{" characters in regular expression patterns are no longer permissible

You have to now say something like "\{" or "[{]" to specify to match a LEFT CURLY BRACKET; otherwise, it is a fatal pattern compilation error. This change will allow future extensions to the language.

These have been deprecated since v5.16, with a deprecation message raised for some uses starting in v5.22. Unfortunately, the code added to raise the message was buggy and failed to warn in some cases where it should have. Therefore, enforcement of this ban for these cases is deferred until Perl 5.30, but the code has been fixed to raise a default-on deprecation message for them in the meantime.

Some uses of literal "{" occur in contexts where we do not foresee the meaning ever being anything but the literal, such as the very first character in the pattern, or after a "|" meaning alternation. Thus

 qr/{fee|{fie/

matches either of the strings {fee or {fie. To avoid forcing unnecessary code changes, these uses do not need to be escaped, and no warning is raised about them, and there are no current plans to change this.

But it is always correct to escape "{", and the simple rule to remember is to always do so.

See Unescaped left brace in regex is illegal here.

scalar(%hash) return signature changed

The value returned for scalar(%hash) will no longer show information about the buckets allocated in the hash. It will simply return the count of used keys. It is thus equivalent to 0+keys(%hash).

A form of backward compatibility is provided via Hash::Util::bucket_ratio() which provides the same behavior as scalar(%hash) provided in Perl 5.24 and earlier.

keys returned from an lvalue subroutine

keys returned from an lvalue subroutine can no longer be assigned to in list context.

    sub foo : lvalue { keys(%INC) }
    (foo) = 3; # death
    sub bar : lvalue { keys(@_) }
    (bar) = 3; # also an error

This makes the lvalue sub case consistent with (keys %hash) = ... and (keys @_) = ..., which are also errors. [perl #128187]

The ${^ENCODING} facility has been removed

The special behaviour associated with assigning a value to this variable has been removed. As a consequence, the the encoding manpage pragma's default mode is no longer supported. If you still need to write your source code in encodings other than UTF-8, use a source filter such as the Filter::Encoding manpage on CPAN or the encoding manpage's Filter option.

POSIX::tmpnam() has been removed

The fundamentally unsafe tmpnam() interface was deprecated in Perl 5.22 and has now been removed. In its place, you can use, for example, the the File::Temp manpage interfaces.

require ::Foo::Bar is now illegal.

Formerly, require ::Foo::Bar would try to read /Foo/Bar.pm. Now any bareword require which starts with a double colon dies instead.

Literal control character variable names are no longer permissible

A variable name may no longer contain a literal control character under any circumstances. These previously were allowed in single-character names on ASCII platforms, but have been deprecated there since Perl 5.20. This affects things like $\cT, where \cT is a literal control (such as a NAK or NEGATIVE ACKNOWLEDGE character) in the source code.

NBSP is no longer permissible in \N{...}

The name of a character may no longer contain non-breaking spaces. It has been deprecated to do so since Perl 5.22.


Deprecations

String delimiters that aren't stand-alone graphemes are now deprecated

For Perl to eventually allow string delimiters to be Unicode grapheme clusters (which look like a single character, but may be a sequence of several ones), we have to stop allowing a single character delimiter that isn't a grapheme by itself. These are unlikely to exist in actual code, as they would typically display as attached to the character in front of them.

\cX that maps to a printable is no longer deprecated

This means we have no plans to remove this feature. It still raises a warning, but only if syntax warnings are enabled. The feature was originally intended to be a way to express non-printable characters that don't have a mnemonic (\t and \n are mnemonics for two non-printable characters, but most non-printables don't have a mnemonic.) But the feature can be used to specify a few printable characters, though those are more clearly expressed as the printable itself. See http://www.nntp.perl.org/group/perl.perl5.porters/2017/02/msg242944.html.


Performance Enhancements


Modules and Pragmata

Updated Modules and Pragmata


Documentation

New Documentation

the perldeprecation manpage

This file documents all upcoming deprecations, and some of the deprecations which already have been removed. The purpose of this documentation is two-fold: document what will disappear, and by which version, and serve as a guide for people dealing with code which has features that no longer work after an upgrade of their perl.

Changes to Existing Documentation

We have attempted to update the documentation to reflect the changes listed in this document. If you find any we have missed, send email to perlbug@perl.org.

Additionally, all references to Usenet have been removed, and the following selected changes have been made:

the perlfunc manpage

the perlguts manpage

the perlhack manpage

the perlhacktips manpage

the perlinterp manpage

the perllocale manpage

the perlmod manpage

the perlmodlib manpage

the perlobj manpage

the perlootut manpage

the perlop manpage

the perlre manpage

the perlretut manpage

the perlunicode manpage

the perlvar manpage


Diagnostics

New Diagnostics

New Errors

New Warnings

Changes to Existing Diagnostics


Utility Changes

c2ph and pstruct

Porting/pod_lib.pl

Porting/sync-with-cpan

perf/benchmarks

Porting/checkAUTHORS.pl

t/porting/regen.t

utils/h2xs.PL

the perlbug manpage


Configuration and Compilation


Testing

Tests were added and changed to reflect the other additions and changes in this release. Furthermore, these substantive changes were made:


Platform Support

New Platforms

NetBSD/VAX
Perl now compiles under NetBSD on VAX machines. However, it's not possible for that platform to implement floating-point infinities and NaNs compatible with most modern systems, which implement the IEEE-754 floating point standard. The hexadecimal floating point (0x...p[+-]n literals, printf %a) is not implemented, either. The make test passes 98% of tests.

Platform-Specific Notes

Darwin
EBCDIC
Several tests have been updated to work (or be skipped) on EBCDIC platforms.

HP-UX
The the Net::Ping manpage UDP test is now skipped on HP-UX.

Hurd
The hints for Hurd have been improved, enabling malloc wrap and reporting the GNU libc used (previously it was an empty string when reported).

VAX
VAX floating point formats are now supported on NetBSD.

VMS
Windows
Linux
Drop support for Linux a.out executable format. Linux has used ELF for over twenty years.

OpenBSD 6
OpenBSD 6 still does not support returning pid, gid, or uid with SA_SIGINFO. Make sure to account for it.

FreeBSD
t/uni/overload.t: Skip hanging test on FreeBSD.

DragonFly BSD
DragonFly BSD now has support for setproctitle(). [perl #130068].


Internal Changes


Selected Bug Fixes


Known Problems


Errata From Previous Releases


Obituary

Jon Portnoy (AVENJ), a prolific Perl author and admired Gentoo community member, has passed away on August 10, 2016. He will be remembered and missed by all those who he came in contact with, and enriched with his intellect, wit, and spirit.

It is with great sadness that we also note Kip Hampton's passing. Probably best known as the author of the Perl & XML column on XML.com, he was a core contributor to AxKit, an XML server platform that became an Apache Foundation project. He was a frequent speaker in the early days at OSCON, and most recently at YAPC::NA in Madison. He was frequently on irc.perl.org as ubu, generally in the #axkit-dahut community, the group responsible for YAPC::NA Asheville in 2011.

Kip and his constant contributions to the community will be greatly missed.


Acknowledgements

Perl 5.26.0 represents approximately 13 months of development since Perl 5.24.0 and contains approximately 360,000 lines of changes across 2,600 files from 86 authors.

Excluding auto-generated files, documentation and release tools, there were approximately 230,000 lines of changes to 1,800 .pm, .t, .c and .h files.

Perl continues to flourish into its third decade thanks to a vibrant community of users and developers. The following people are known to have contributed the improvements that became Perl 5.26.0:

Aaron Crane, Abigail, Ævar Arnfjörð Bjarmason, Alex Vandiver, Andreas König, Andreas Voegele, Andrew Fresh, Andy Lester, Aristotle Pagaltzis, Chad Granum, Chase Whitener, Chris 'BinGOs' Williams, Chris Lamb, Christian Hansen, Christian Millour, Colin Newell, Craig A. Berry, Dagfinn Ilmari Mannsåker, Dan Collins, Daniel Dragan, Dave Cross, Dave Rolsky, David Golden, David H. Gutteridge, David Mitchell, Dominic Hargreaves, Doug Bell, E. Choroba, Ed Avis, Father Chrysostomos, François Perrad, Hauke D, H.Merijn Brand, Hugo van der Sanden, Ivan Pozdeev, James E Keenan, James Raspass, Jarkko Hietaniemi, Jerry D. Hedden, Jim Cromie, J. Nick Koston, John Lightsey, Karen Etheridge, Karl Williamson, Leon Timmermans, Lukas Mai, Matthew Horsfall, Maxwell Carey, Misty De Meo, Neil Bowers, Nicholas Clark, Nicolas R., Niko Tyni, Pali, Paul Marquess, Peter Avalos, Petr Písař, Pino Toscano, Rafael Garcia-Suarez, Reini Urban, Renee Baecker, Ricardo Signes, Richard Levitte, Rick Delaney, Salvador Fandiño, Samuel Thibault, Sawyer X, Sébastien Aperghis-Tramoni, Sergey Aleynikov, Shlomi Fish, Smylers, Stefan Seifert, Steffen Müller, Stevan Little, Steve Hay, Steven Humphrey, Sullivan Beck, Theo Buehler, Thomas Sibley, Todd Rinaldo, Tomasz Konojacki, Tony Cook, Unicode Consortium, Yaroslav Kuzmin, Yves Orton, Zefram.

The list above is almost certainly incomplete as it is automatically generated from version control history. In particular, it does not include the names of the (very much appreciated) contributors who reported issues to the Perl bug tracker.

Many of the changes included in this version originated in the CPAN modules included in Perl's core. We're grateful to the entire CPAN community for helping Perl to flourish.

For a more complete list of all of Perl's historical contributors, please see the AUTHORS file in the Perl source distribution.


Reporting Bugs

If you find what you think is a bug, you might check the perl bug database at https://rt.perl.org/. There may also be information at http://www.perl.org/, the Perl Home Page.

If you believe you have an unreported bug, please run the the perlbug manpage program included with your release. Be sure to trim your bug down to a tiny but sufficient test case. Your bug report, along with the output of perl -V, will be sent off to perlbug@perl.org to be analysed by the Perl porting team.

If the bug you are reporting has security implications which make it inappropriate to send to a publicly archived mailing list, then see SECURITY VULNERABILITY CONTACT INFORMATION in the perlsec manpage for details of how to report the issue.


Give Thanks

If you wish to thank the Perl 5 Porters for the work we had done in Perl 5, you can do so by running the perlthanks program:

    perlthanks

This will send an email to the Perl 5 Porters list with your show of thanks.


SEE ALSO

The Changes file for an explanation of how to view exhaustive details on what changed.

The INSTALL file for how to build Perl.

The README file for general stuff.

The Artistic and Copying files for copyright information.

 perl5260delta - what is new for perl v5.26.0