Seven Days between the Parrot and the Camel

from Monday, December 21 to Sunday, December 27 in the year 2009

Considering the Language:

The spec remained quiet during Christmas.

♢

Considering Rakudo:

Apart from some core setting work by Solomon Foster and David Romano, the ng branch was silent.

The master branch learned :s for getting file sizes.

♢

Considering Parrot:

Work is going on to make PIRC easier to build. (PIRC is an alternative PIR compiler with a yacc/bison grammar and a flex-generated lexer. It is meant to replace the IMCC compiler.) The pirc_config branch has been created for adding a --pirc configure option. The goal is to make using PIRC an option before Parrot 2.0.

Will Coleda (a.k.a coke) improved the optional automatic testing of make dependencies (tools/dev/checkdepend.pl) and fixed many of the reported errors.

Christoph Otto cleaned up the description of the bytecode format in PDD 13 and made packfiles somewhat smaller and faster by omitting redundant pc → filename mappings.

Peter Lobsinger and Kevin Tew continued work in the pmc_freeze_cleanup branch that aims at cleaning up and unifying PMC freeze/thaw serialization and packfile reading/writing.

Geoffrey Broadwell has completed the migration of plumage glue code to nqp-rx. All regexes are written in nqp-rx now, so PGE is no longer required. All plumage-specific code has been removed from Util.nqp in order to prepare the file for possible inclusion as a library in the nqp-rx distribution. However, Util.nqp still needs tests and a ticket has been opened in order to encourage contributors. There is also a document explaining how to add tests to plumage.

[As mistakenly reported for week 51,] Bob Kuo, a.k.a. bubaflub, celebrated his new commit bit with a firework of changes, converting many further tests to PIR among other things.

♢

Seven Days between the Parrot and the Camel

from Monday, December 14 to Sunday, December 20 in the year 2009

Considering the Language:

The Cool class appears, an ancestor for all classes that are willing to convert their values in order to support the "culturally universal" methods defined in Cool (for example .abs or .substr). The mutable Cool namespace will also contain the multi methods of last resort that are searched if normal multiple dispatch does not find a viable candidate. All built-in classes except junctions and Whatever derive from Cool. User-defined classes do not derive from Cool by default.

The verbs for leaving a when block early will be proceed and succeed (instead of the false friends continue and break). proceed resumes execution after the closing curly of the when block. succeed immediately leaves the surrounding topicalizer block.

The loose prefix operator for testing truth has been renamed from true to so in order to make it more distinct from True and shorter than not.

The REJECTS method has been removed. The negated smart match operator !~~ will use !foo.ACCEPTS instead.

Picking from baggy containers has been refined. A separate .grab method will do a mutating .pick which removes the selected element (once) from the container. When used without replacement, .pick will keep a private temporary copy of the Bag and .grab from that. The .pickpairs and .grabpairs methods will treat each key/replication count pair as a single item to pick (and remove).

♢

Considering Rakudo:

Shortly after Parrot's 1.9.0 release, the Rakudo master branch was released as development release #24 ("Seoul"). Due to spectest changes, this release passes fewer tests than #23, but the relative percentage of passed tests has increased. The most visible changes in #24 are the switch from Object to Mu and the removal of undef.

In the ng branch, Jonathan Worthington reimplemented does in terms of the meta-model, not relying on Parrot's primitives. Work continued to make more spectest pass with the ng branch.

♢

Considering Parrot:

Gerd Pokorra released Parrot 1.9.0 "Blue-fronted Amazon" on December 15. Among the most notable changes are the greatly improved nqp-rx compiler and the extensible profiling runcore.

The Parrot developers held a roadmap meeting, identifying priorities for upcoming Parrot milestones. An important outcome was the change to a shorter 3-month support cycle. Parrot development for 2.0 and 2.3 will focus on the needs of Rakudo *.

The context_unify3_simple branch by Vasily Chekalkin landed and merged the Context and CallSignature PMCs into CallContext. The branch is a descendant of context_unify3 that omits some more involved changes to context creation.

In the pmc_freeze_cleanup branch created by Peter Lobsinger, the Parrot developers are busily cleaning up and merging freeze/thaw serialization and the PBC bytecode format. The new code uses the visit_info struct as its pivot and keeps out of string internals.

François Perrad continued to improve distutils and languages infrastructure.

Bob Kuo, a.k.a. bubaflub, celebrated his new commit bit with a firework of changes, converting many further tests to PIR among other things.

♢

Seven Days between the Parrot and the Camel

from Monday, December 7 to Sunday, December 13 in the year 2009

Considering the Language:

Role composition rules changed so the collision of has attributes with the same name will always be considered a conflict, even if their types happen to match.

The p{...} quoting construct for filenames has been removed in favour of the :p modifier, so paths will be written as q:p{...}, or qp{...} in short.

Bare say and print were reduced from errors to warnings. Upon a question about useful defaults, Larry Wall explained that Perl 6 will free the programmer from memorizing arbitrary lists of special cases. He also pointed out the concise forms .say and .print for printing the topic.

♢

Considering Rakudo:

On the Rakudo ng branch, Jonathan Worthington continued work on the meta-object protocol, filling in more parts of role composition. Attributes are now properly composed into packages by calling the compose method on the meta-attribute. For initialized attributes, a build closure is passed along. The RoleHOW meta-class moved towards independence from Parrot's Role PMC.

Notably, the does and but operators for mixing in roles are now written entirely in Perl 6, and the role appliers are NQP classes with only a little Parrot-specific code.

Jonathan Worthington also added a first sketch of an implementation of enumeration types.

Solomon Foster, Moritz Lenz, and Stephen Weeks did more work on the core setting. As of this writing, the ng branch passes over 2400 spectests.

In the nqp-rx repository, Patrick Michaud started to rewrite the HLL::Compiler class in NQP.

♢

Considering Parrot:

Christoph Otto refactored the profiling runcore to allow multiple output formats. Currently it supports the human-readable pprof format and the silent none format (for profiling the runcore itself). Plans are to add binary output and output in callgrind format. The updated runcore also supports recording annotations in pprof files. Environment variables are used to control these features.

Working on the last part of his Hague grant, Patrick Michaud committed a new draft for PDD 31 on HLL compilers and interoperability.

The running of :immediate, non-:anon subs is now delayed when writing PBC bytecode, so the subs can run during reading PBC, which should make .const more usable for HLLs (as :immediate subs may set up things for creating the constant data from the bytecode).

Vasily Chekalkin created the context_unify3 branch in the third attempt to merge the CallSignature and Context PMC types. The unified PMC will be known as CallContext.

Bob Rogers posted a proof-of-concept patch for unwinding nested runloops upon the invocation of an exception handler.

Peter Lobsinger is looking into merging the PBC bytecode format and the freeze/thaw serialization format for PMCs.

The December 12 pre-release hackathon focused on removing uses of the strstart field outside the string internals. However, many occurrences cannot be replaced easily, and Andy Dougherty and Patrick Michaud warned about breaking code for cases of substrings that may not be tested.

A lot of cage cleaning happened this week: JimmyZ contributed many patches to make PMC code use GET_ATTR and SET_ATTR macros. Bubaflub converted further tests to PIR. François Perrad improved C code indentation in areas not yet tested by codingstd.t. Michael H. Hind fixed g++ build failures.

Geoffrey Broadwell added Mercurial support to plumage and documented how to add new build tools to the system.

♢

Seven Days between the Parrot and the Camel

from Monday, November 30, to Sunday, December 6, in the year 2009

Considering the Language:

Enumeration types will be collections of Enums, that is of pairs with constant values (formerly known as PairVals). Like a Pair, an Enum has .key and .value, but both are read-only. An (anonymous) enum declaration creates an EnumMap, a sequence of Enums, the values of which are determined at compile time. An .enums accessor will be provided by enumeration types to get at the associations (instead of .mapping), and by other containers as a read-only analog to .pairs.

The collection types EnumMap and its modifiable value analog PairMap (a sequence of Pairs) will do roles for both positional and associative access. At the implementation's discretion, the latter may use anything from linear scan to optimized hash-like lookup.

The mixins created with but fall into two categories: Mixing in an enumeration type adds a modifiable attribute. Mixing in a value adds a read-only accessor method. The attribute added for rw mixins starts out uninitialized, unless a WHENCE closure is given.

♢

Considering Rakudo:

While the master branch remained quiet except for an improvement of Rat operators, the ng branch continued to be the focus of development. In the course of his work on the meta-object protocol, Jonathan Worthington implemented role to instance application. Stephen Weeks committed a first draft of CATCH and CONTROL. Solomon Foster and Moritz Lenz restored many parts of the core setting, and perlpilot added while and until statement modifiers.

Stephen Weeks plans to implement lazy gather and take, but is blocking on the redesign of Rakudo's iterator interface.

Meanwhile, nqp-rx acquired an initial version of the smartmatch operator, the /.../ construct for writing regexes and improved longname syntax.

♢

Considering Parrot:

Garbage collection gained further attention when Patrick Michaud posted an observation about heavy GC overhead which he presumed to be caused by a small number of mortal but long-lived objects and inefficient marking. Contrariwise, chromatic attributed the overhead to the large number of short-lived objects generated internally by the Parrot Calling Conventions (PCC). Both supported their arguments with interesting experimental data.

Various approaches were suggested, among them separate pools for immortal and PCC objects, combining objects to superobjects with escape pointer lists, stack allocation, and avoiding needless autoboxing.

Traces of recycled objects showed pointless GC runs during startup, which led to an adjustment of the replenish level factor.

Vasily Chekalkin merged the CallSignature and CallSignatureReturns PMC types in order to reduce the number of objects created. This change improved the fib.pir benchmark by 7%.

PCC learned to use the HLL type map when autoboxing values.

Patrick Michaud questioned the deprecation of label-based exception handlers, which led to discussions of continuations invoked as/by exception handlers and the resulting inferior runloop problem. [To the reporter's understanding this problem arises when an exception handler declared in one runloop is invoked in another (nested) runloop and then resumes execution by falling through to the surrounding code.] The general consensus seemed to be that while nested runloops should be removed on the long term, an intermediate solution — e.g. an explicit opcode for resuming execution in the runloop declaring the handler — was needed.

Jonathan Leto provided microbenchmark results for past and present parrot versions.

The question was raised whether PIR should get an __END__ tag, but the response was mostly negative.

François Perrad committed many additions to and refactorings of distutils.

Geoffrey Broadwell reports that 80% of plumage refactorings are done and that it is time to add new features again.

♢

Seven Days between the Parrot and the Camel

from Monday, November 23, to Sunday, November 29, in the year 2009

Considering the Language:

Seeking a word that conveys both allness and nothingness, Larry Wall renamed the root of the Perl 6 type hierarchy to Mu, an allusion to the Zen concept of "no thing" or emptiness as the nature of all things. (Cows were also mentioned.)

The radical change to Mu was the outcome of a wider exploration: undef as a term becomes obsolete, as there will be no singleton undefined value in Perl 6, but rather various flavours, for example the most undefined Mu, other protoobjects carrying type information, failure objects (created with fail), or Nil.

Said Nil is no longer a type, but a name for the empty parcel (), which represents the absence of return values and resets containers to their initial value upon assignment (Mu for untyped scalar containers).

The sink prefix dissolves returned values to Nil, which can help trigger actions in case of empty lists as in "for @some || sink { warn "None!" } {...}".

The convenience macro .notdef and multi notdef will be useful for matching undefined values.

The chapter about enums in Synopsis 12 got expanded. Enum types are no longer roles, instead the types of enum values will just be guaranteed to be derived from and convertible to the enum's base type. Enum keys may be used as type names (each type containing a single enum value). The enum type itself appears like a typename package containing a set of constant declarations and supplies a .mapping method to get at the translation of symbolic values to enum values.

♢

Considering Rakudo:

Patrick Michaud's reimplementation of "Not Quite Perl", nqp-rx, kept improving: Quoted strings can now be used as method names (including interpolation), regexes learned to heed :pos and :continue options, and Stephen Weeks contributed exception handling via try, CATCH, and CONTROL.

The rakudo ng branch continued climbing up the spectest curve, which turned out steeper than initially hoped for, as many test files require features beyond those actually targeted by the respective test.

Still the branch made good progress, as the Rakudo developers added eval, lazy signature binding, regexes, and parts of the meta-object protocol, which was the focus of Jonathan Worthington's Rakudo Day.

♢

Considering Parrot:

Last week, chromatic had observed that in one of Rakudo's use cases the repeated marking of a modest number of long-lived objects consumed over 88% of total CPU time. This finding led to discussions about garbage collection this week, and to the general agreement that a generational garbage collector would be an important improvement. Allison Randal confirmed that garbage collection will become a likely focus of development after release 2.0.

Cotto committed an NQP port of the profiling script pprof2cg, noting, however, that it was 26.5 times slower than the Perl version.

Patches by bubaflub and kurahaupo converted further tests from Perl 5 to PIR.

Allison Randal sent a note about MMD to the parrot-dev list, outlining how hypercube graphs could be used for fast calculation of Manhattan distances during dispatch.

Plumage saw a lot of refactoring and glue code reduction as more features of nqp-rx became available. Geoffrey Broadwell proposed a naming scheme for plumage dependencies which seemed to be well received. François Perrad continued work on his distutils package and contributed plumage metadata for several projects. As of this writing, the metadata base of plumage counts 23 projects.

♢

Seven Days between the Parrot and the Camel

from Monday, November 16, to Sunday, November 22, in the year 2009

Considering Parrot:

The Parrot trunk remained mostly quiet after release 1.8.0 went out, while the Parrot developers finished the move from rt.perl.org to the Trac issue tracker in a burst of activity. James Keenan, in particular, did a huge amount of ticket updates and documentation fixes.

Most notably, nqp-rx, Patrick Michaud's self-hosting reimplementation of "Not Quite Perl" and Perl 6 regexes, is now distributed along with Parrot and gaining new users. Among them is plumage which became the source of several nqp-rx feature requests.

François Perrad's build system "distutils" saw many additions, among them an update step, smoke target, plumage target, and a Win32 Inno installer target.

Similar to PMCNULL, STRINGNULL is now available for representing the lack of a STRING.

Addressing the needs of HLL developers, chromatic implemented several performance improvements, squeezing over 12% [reporter's calculation] out of the nqp-rx Action.pm benchmark. Most of this is due to the addition of a method cache in the Object PMC and special-casing of hot paths in the Hash PMC.

A new mailing list parrot-users (with group) was installed so language authors can seek assistance undisturbed by the sounds of Parrot's internal machinery.

♢

Considering Rakudo:

Development release #23 "Lisbon" was cut from master, but most of this weeks activity took place on the new "ng" branch where Patrick Michaud and Jonathan Worthington are quickly rebuilding Rakudo's internals using nqp-rx. As more features become available, contributors are starting to implement parts of the core setting in Perl 6.

♢

Considering the Language:

The seventh meta-operator S appears with the ability to attach a sequence point to any non-fiddly infix operator. The resulting operator guarantees that its operands are evaluated sequentially and that shortcuts are taken where applicable. Former also is now spelled S&, and the junctive sequential else -- which only made a brief occurrence -- is spelled S|.

An area that might still need some sorting out is the combination of the sequencing S with the reversal meta-operator R. Conjecturally, application of R reverses the sequence of evaluation along with the operands, i.e. a RS/ b evaluates a first and b second, while a SR/ b evaluates b first and a second.

The semantics of rational numbers were elaborated and balanced between precision and performance. FatRats joined the pack providing arbitrary precision in both numerator and denominator at any cost. The parametric Rational role serves as the general abstraction for rational numbers, analogous to Integral for integers.

The stringification prefix ~ got divorced from .Str. Unary ~ imposes a Stringy context and -- like Perl 5 stringification --, will turn undefined values into the empty string (with warning).

Slightly changing, constant now considers itself a type declarator, declaring a type that has a single value determined at BEGIN time (so the optimizer can inline its uses). Scope declarators as in my constant, has constant, etc. can be used to specify visibility and storage location. By default constants (like enums) get our scope.

A new scope declarator, anon, can be used to create anonymous subs with "of" type (i.e. anon Foo sub {...}), so you don't have to abuse my or our for this purpose.

The "void" context got tentatively renamed to "sink". (Like its big sister, Perl 6 has three main contexts (sink, item, and list) controlling whether an expression should yield none, one, or many of its values.)

Parcel semantics got some attention this week and the is ref parameter trait changed to is parcel. Larry encourages us to spell \$p instead of $p is parcel, however, as the trait is mainly meant for introspection. (As every language feature needs its antagonist, Parcels show up when context is not called for. They provide a way for passing data without imposing context.)

In other cont...areas the heavily overloaded word "context" was allowed to drop some of its burden: "contextual" variables became "dynamic" variables and are now accessible via the DYNAMIC pseudo-package. Things formerly known as "dynamic contexts" or "activation records" are now simply "call frames".

♢