jump to navigation

So, what about (Perl 6) dependencies? 2017-05-28

Posted by claudio in Uncategorized.
Tags: , , , ,
9 comments

DependenciesWhen I need to program something, most of the time I use Perl 5, Go or Perl 6. Which one depends on the existence and maturity of libraries and the deployment strategy (and I must admit, probably my mood). Most applications I write at work are not that big, but they need to be stable and secure. Some end up in production as an extension or addition to the software that is the core of our authentication and authorisation infrastructure. Some programs are managed by other teams, e.g. of sysadmin-type applications like the monitoring of a complex chain of microservices. Finally, proof of concept code is often needed when designing a new architecture. I believe that If your software is cumbersome/fragile to install and maintain, people won’t use it. Or worse, stick with an old version.

Hence, I like CPAN. A lot. I love how easy it is to download, build, test and install Perl 5 libraries. cpanminus made the process even more zero-conf. When not using Docker, tools like App::Fatpacker and Carton can bundle libraries and you end with a mostly self-contained application (thx mst and miyagawa).

Go, a compiled language, took a different path and opted for static binaries. The included (Go) libraries are not downloaded and built when you’re deploying, but when you’re developing. Although this is a huge advantage, I am not too fond of the dependency system: you mostly end up downloading random versions from the master branch of random Github repos (the workaround is using external webservices like gopkg.in, not ideal). The only sane way is to “vendor in” (also with a tool) your dependencies pretty much the same way Carton does: you copy the libs in your repository (but still without versioning). I hear the Go community is working at this, but so far there are only workarounds.

In the Perl 6 world, the zef module manager does provide a kind of cpanminus-like experience. However, in contrast with cpanminus it does this by downloading code from a zillion Github repo’s where the versioning is questionable. The is no clear link between the version fixed in the metadata (META6.json) and branches/tags on the repo. Like mentioned above, Go gets away with this due to static compiling, although the price is high: your projects will have dozens of versions of the same lib, probably even with a different API… and no way to declare the version in the code.

The centralised Perl 5 approach approach is fairly complex. It works because of the maturity of the ecosystem (the “big” modules are pretty stable) and the taken-for-granted testing culture (thank you toolchain and testing people!). Actually, in my opinion, the only projects that really solved the dependency management problem are the Linux & BSD distributions by boxing progress in long release cycles. Developers want the last shiny lib, so that won’t work.

The Perl 6 devs have no static compiling on the agenda, so it’s clear that the random Github repo situation is far from ideal. That’s why I was pretty excited to read on #perl6 that Perl 6 code can now be uploaded to CPAN (with Shoichi Kaji’s mi6) and installed (with Nick Logan’s zef). Today, the Perl 6 ecosystem has neither the maturity of the one of Perl 5 nor its testing culture. The dependency chain can be pretty fragile at times. Working within a central repository with an extensive and mature testing infrastructure will certainly help over time. One place to look for libraries, rate them, find the documentation, and so on. Look at their state and the state of its dependencies. This is huge. But I don’t think that CPAN will fix the problems of a young ecosystem right away. I think there is an opportunity here to build on the shoulders on CPAN, while keeping the advantages of a new language: find out what works.

Personally, I would love to have a built-in packaging solution like Carton –or even App::Fatpacker– out of the box. I think there is something to be said for the “vendoring-in” of the dependencies in the application repo (putting all the dependencies in a directory in the repo). The Perl 6 language/compiler has advantages over Perl 5. You can specify the compiler you target so your code doesn’t break when your compiler (also) targets a more recent milestone (allowing the core devs to advance by breaking stuff). Soon, you’ll be able to You can even load different versions of the same library in the same program (thx for the correction, nine).

The same tool could even vendor (and update) different version of the same library. At “package time” this tool would look at your sources and include only the versions of the libraries that are referenced. The result could be a tar or a directory with everything needed to run your application. As a bonus point, it would be nice to still support random repos for libraries-in-progress or for authors that opted not to use CPAN.

My use case is not everyone’s, so I wonder what people would like to see based on their experience or expectations. I think that now, with the possibility of a CPAN migration, it a good time to think about this. Let’s gather ideas. The implementation is for later :).

Advertisements

rakudo-pkg: Create OS packages for Rakudo Perl 6 using Docker 2016-09-05

Posted by claudio in Uncategorized.
Tags: , , , , ,
1 comment so far

camelia-logo

There was an interesting discussion on #perl6 (irc.freenode.net) about the use of rakudobrew as a way for end-users to install Rakudo Perl 6 (see how-to-get-rakudo).

rakudobrew, inspired by perlbrew, is a way to manage (and compile) different versions of rakudo. nine argued that it’s primarily meant as a tool for rakudo developers. Because of the increased complexity (e.g. when dealing with modules) it’s not targeted at end-users. While being a big fan of rakudobrew, I agree with nine.

The problem is that there are no Linux binaries on the download page (there are for MacOS and Windows), so users are stuck with building from source (it can be fun, but after a while it isn’t).

rakudo-pkg is a github project to help system administrators (and hopefully Rakudo release managers) to easily provide native Linux packages for end users. So far, I added support for creating Ubuntu 16.04 LTS amd64 and i386 packages and Centos 7 amd64. These are the systems I use the most. Feel free to add support for more distributions.

rakudo-pkg uses Docker. The use of containers means that there is no longer need for chasing dependencies and no risks of installing files all over your system. It also means that as long the building machine is a Linux 64-bit OS, you can build packages for *all* supported distributions.

Within the containers, rakudo-pkg uses fpm. The created packages are minimalistic by design: they don’t run any pre/post scripts and all the files are installed in /opt/rakudo. You’ll have to add /opt/rakudo/bin to your PATH. I also added two additional scripts to install Perl 6 module managers (both have similar functionalities):

install_panda_as_user.sh
install_zef_as_user.sh

If you just want to create native packages, just go to the bin directory and execute the run_pkgrakudo.pl command. In this case there is no need to locally build the Docker images: you’ll automatically retrieve the image from the rakudo namespace on Docker Hub. Of course, if you want to create the container images locally, you can use the supplied dockerfiles in the docker directory. Have a look at the README.md for more information.

You can find examples of packages created with rakudo-pkg here (they need to be moved to a more definitive URL).

Have fun.

https://github.com/nxadm/rakudo-pkg

Vim as a Perl 6 editor 2016-08-21

Posted by claudio in Uncategorized.
Tags: , ,
2 comments

EDITED on 20170211: syntastic-perl6 configuration changes

If you’re a Vim user you probably use it for almost everything. Out of the box, Perl 6 support is rather limited. That’s why many people use editors like Atom for Perl 6 code.

What if with a few plugins you could configure vim to be a great Perl 6 editor? I made the following notes while configuring Vim on my main machine running Ubuntu 16.04. The instructions should be trivially easy to port to other distributions or Operating Systems. Skip the applicable steps if you already have a working vim setup (i.e. do not overwrite you .vimrc file).

I maintain my Vim plugins using pathogen, as it allows me to directly use git clones from github. This is specially important for plugins in rapid development.
(If your .vim directory is a git repository, replace ‘git clone’ in the commands by ‘git submodule add’.)

Basic vim Setup

Install vim with scripting support and pathogen. Create the directory where the plugins will live:
$ sudo apt-get install vim-nox vim-pathogen && mkdir -p ~/.vim/bundle

$ vim-addons install pathogen

Create a minimal .vimrc in your $HOME, with at least this configuration (enabling pathogen). Lines commencing with ” are comments:

“Enable extra features (e.g. when run systemwide). Must be before pathogen
set nocompatible

“Enable pathogen
execute pathogen#infect()
“Enable syntax highlighting
syntax on
“Enable indenting
filetype plugin indent on

Additionally I use these settings (the complete .vimrc is linked atthe end):

“Set line wrapping
set wrap
set linebreak
set nolist
set formatoptions+=l

“Enable 256 colours
set t_Co=256

“Set auto indenting
set autoindent

“Smart tabbing
set expandtab
set smarttab
set sw=4 ” no of spaces for indenting
set ts=4 ” show \t as 2 spaces and treat 2 spaces as \t when deleting

“Set title of xterm
set title

” Highlight search terms
set hlsearch

“Strip trailing whitespace for certain type of files
autocmd BufWritePre *.{erb,md,pl,pl6,pm,pm6,pp,rb,t,xml,yaml,go} :%s/\s\+$//e

“Override tab espace for specific languages
autocmd Filetype ruby,puppet setlocal ts=2 sw=2

“Jump to the last position when reopening a file
au BufReadPost * if line(“‘\””) > 1 && line(“‘\””) <= line(“$”) |
\ exe “normal! g’\”” | endif

“Add a coloured right margin for recent vim releases
if v:version >= 703
set colorcolumn=80
endif

“Ubuntu suggestions
set showcmd    ” Show (partial) command in status line.
set showmatch  ” Show matching brackets.
set ignorecase ” Do case insensitive matching
set smartcase  ” Do smart case matching
set incsearch  ” Incremental search
set autowrite  ” Automatically save before commands like :next and :make
set hidden     ” Hide buffers when they are abandoned
set mouse=v    ” Enable mouse usage (all modes)

Install plugins

vim-perl for syntax highlighting:

$ git clone https://github.com/vim-perl/vim-perl.git ~ /.vim/bundle/vim-perl

vim-perl

vim-airline and themes for a status bar:
$ git clone https://github.com/vim-airline/vim-airline.git ~/.vim/bundle/vim-airline
$ git clone https://github.com/vim-airline/vim-airline-themes.git ~/.vim/bundle/vim-airline-themes
In vim type :Helptags

In  Ubuntu the ‘fonts-powerline’ package (sudo apt-get install fonts-powerline) installs fonts that enable nice glyphs in the statusbar (e.g. line effect instead of ‘>’, see the screenshot at https://github.com/vim-airline/vim-airline/wiki/Screenshots.

Add this to .vimrc for airline (the complete .vimrc is attached):
“airline statusbar
set laststatus=2
set ttimeoutlen=50
let g:airline#extensions#tabline#enabled = 1
let g:airline_theme=’luna’
“In order to see the powerline fonts, adapt the font of your terminal
“In Gnome Terminal: “use custom font” in the profile. I use Monospace regular.
let g:airline_powerline_fonts = 1

airline

Tabular for aligning text (e.g. blocks):
$ git clone https://github.com/godlygeek/tabular.git ~/.vim/bundle/tabular
In vim type :Helptags

vim-fugitive for Git integration:
$ git clone https://github.com/tpope/vim-fugitive.git ~/.vim/bundle/vim-fugitive
In vim type :Helptags

vim-markdown for markdown syntax support (e.g. the README.md of your module):
$ git clone https://github.com/plasticboy/vim-markdown.git ~/.vim/bundle/vim-markdown
In vim type :Helptags

Add this to .vimrc for markdown if you don’t want folding (the complete .vimrc is attached):
“markdown support
let g:vim_markdown_folding_disabled=1

synastic-perl6 for Perl 6 syntax checking support. I wrote this plugin to add Perl 6 syntax checking support to synastic, the leading vim syntax checking plugin. See the ‘Call for Testers/Announcement’ here. Instruction can be found in the repo, but I’ll paste it here for your convenience:

You need to install syntastic to use this plugin.
$ git clone https://github.com/scrooloose/syntastic.git ~/.vim/bundle/synastic
$ git clone https://github.com/nxadm/syntastic-perl6.git ~/.vim/bundle/synastic-perl6

Type “:Helptags” in Vim to generate Help Tags.

Syntastic and syntastic-perl6 vimrc configuration, (comments start with “):


"airline statusbar integration if installed. De-comment if installed
"set laststatus=2
"set ttimeoutlen=50
"let g:airline#extensions#tabline#enabled = 1
"let g:airline_theme='luna'
"In order to see the powerline fonts, adapt the font of your terminal
"In Gnome Terminal: "use custom font" in the profile. I use Monospace regular.
"let g:airline_powerline_fonts = 1

“syntastic syntax checking
let g:syntastic_always_populate_loc_list = 1
let g:syntastic_auto_loc_list = 1
let g:syntastic_check_on_open = 1
let g:syntastic_check_on_wq = 0
set statusline+=%#warningmsg#
set statusline+=%{SyntasticStatuslineFlag()}
set statusline+=%*

“Perl 6 support
“Optional comma separated list of quoted paths to be included to -I
“let g:syntastic_perl6lib = [ ‘/home/user/Code/some_project/lib’, ‘lib’ ]
“Optional perl6 binary (defaults to perl6 in your PATH)
“let g:syntastic_perl6_exec = ‘/opt/rakudo/bin/perl6’
“Register the checker provided by this plugin
let g:syntastic_perl6_checkers = [‘perl6’]
“Enable the perl6 checker (disabled out of the box because of security reasons:
“‘perl6 -c’ executes the BEGIN and CHECK block of code it parses. This should
“be fine for your own code. See: https://docs.perl6.org/programs/00-running
let g:syntastic_enable_perl6_checker = 1

Screenshot of syntastic-perl6

You complete me fuzzy search autocomplete:

$ git clone https://github.com/Valloric/YouCompleteMe.git ~/.vim/bundle/YouCompleteMe

Read the YouCompleteMe documentation for the dependencies for your OS and for the switches for additional non-fuzzy support for additional languages like C/C++, Go and so on. If you just want fuzzy complete support for Perl 6, the default is ok. If someone is looking for a nice project, a native Perl6 autocompleter for YouCompleteMe (instead of the fuzzy one) would be a great addition. You can install YouCompleteMe like this:
$ cd ~/.vim/bundle/YouCompleteMe && ./install.py

autocomplete

That’s it. I hope my notes are useful to someone. The complete .vimrc can be found here.

 

 

Please test: first release of syntastic-perl6, a vim syntax checker 2016-08-20

Posted by claudio in Uncategorized.
Tags: , ,
1 comment so far

Vimlogo.svgI think that Perl 6, as a fairly new language, needs good tooling not only to attract new programmers but also to make the job of Perl 6 programmers more enjoyable. If you’ve worked with an IDE before, you certainly agree that syntax checking is one of those things that we take for granted. Syntastic-perl6 is a plugin that adds Perl 6 syntax checking in Vim using Syntastic. Syntastic is the leading Vim plugin for syntax checking. It supports many programming languages.

If the plugin proves to be useful, I plan on a parallel track for Perl 6 support in Vim. On one hand, this plugin will track the latest Perl 6 Rakudo releases (while staying as backwards compatible as possible) and be the first to receive new functionality. On the other hand, once this plugin is well-tested and feature complete, it will hopefully be added to the main syntastic repo (it has it’s own branch upstream already) in order to provide out-of-the-box support for Perl 6.

So, what do we need to get there? We need testers and users, so they can make this plugin better by:

  • sending Pull Requests to make the code (vimscript) better where needed.
  • sending Pull Requests to add tests for error cases not yet tested (see the t directory) or -more importantely- caught.
  • posting issues for bugs or errors not-yet-caught. In that case copy-paste the error (e.g. within vim: :!perl6 -c %) and post a sample of the erroneous Perl 6 code in question.

The plugin, with installation instructions, is on its github repo at syntastic-perl6. With a vim module manage like pathogen you can directly use a clone of the repo.

Keep me posted!