A Lunar Module (Wikipedia)

Just in case nobody noticed, the Module Site has received a much-needed face-lift, which uses the base template for the main Lunar-Linux website. Some minor changes were also made to improve the overall speed and HTML-correctness since some areas actually had broken markup. The site’s URL scheme was also updated to be more search-engine friendly and proper 404 pages/headers were added where needed. The color scheme and table layout remains, but this may change in the near future as I decide where I would like to go with things. Caching and compression is the next priority, as some dependency pages can take longer than wanted to generate and render.

Beyond that, the wiki and other sites may also receive the same treatment. The screenshots site also leaves a lot to be desired since no work went into the template after the Gallery update, but I am considering migrating it into the main website as a sub-blog. There are various good and bad things about that idea, though. If anyone has ideas, suggestions, grievances, etc. please feel free to share them.

Ever been curious about Lunar’s module statistics? Wonder no more.

1. Retain the basic concepts of the current scripts and improve them to handle additional network connection types. In general just make them better than their current state.
2. Switch to wicd.
3. Switch to NetworkManager
4. Switch to connman

Item one sounds the most appealing since another application would not need adding to the ISO. Well that would be true if only the very basic of network connectivity is desired out of the box such as Ethernet and WiFi. Anything more like VPN and PPPoE will of course need adding to the default ISO. Staying with our own scripts does allow us some flexibility. A good example was a thought by Ratler to move away from the use of ifconfig and friends to iputils.

Wicd has some possibilities. I have used it in the past and for the simple setups I needed, it worked fine. However compared to Item three and four I do not know how well it stacks up with more complicated networks. If their Home page is correct it still does not support PPPoE/DSL.

The NetworkManager has never given me a warm fuzzy. Why I cannot say especially since I have never used it; it is what it is. Perhaps it is in the past I have tried to compile it with the aim of adding it to moonbase. Well there is a show stopper. Color me a newb but I cannot get past this configure error;

Your distribution (lunar linux) is not yet supported!  (patches welcome)

Being the bright newb that I am the most obvious thing to try is this; –with-distro=none or some variant; nope. OK, what about –without-distro; nope. Now this confuses me since Mr. William claims it can be turned off; NetworkManager and ConnMan. Perhaps I am being a dunderhead and not seeing the obvious. Let’s suppose that is not the case and a distro name must be supplied. Let me say it is not like our networking scripts change that often (ahem) but would any dev want to fiddle with sending them patches?

An alternative to maintaining a patch set specifically for them to incorporate we could , shudder, make our  /etc directory structure to fit one of the distros they do support. Some of you should know my position about such things as I have responded to some in IRC. On occasion we get a visitor and in short thrift begins to comment/complain why Lunar does not do $SOMETHING like $DISTRO. I need not comment further.

Now on to connman. Compared to NetworkManager it is less distro dependent and we do not need deal with restructuring /etc to fit some $DISTRO. AFAICT it does support more different types of network connections than wicd.

It seems to me we only have two real choices; rewrite our scripts or connman. On the former it could well be that we would need to restructure /etc in some fashion. That would tend to reduce (though not a lot) the negative I see with NetworkManager.

Image by Adriano Gasparri via Flickr

Hopefully it’s quite obvious that the website has changed just a bit. This is because of our migration over to WordPress from Joomla, which was brought about due to a number of factors:

• Easier to maintain once set up
• Easier to keep updated & migrate to new versions
• Better selection of ready-made themes and plugins

Our site has been rather stale for a time, as we had not seen a fresh post since August, 2010 for the release of 1.6.5 however this was largely due to the main site only being used for news announcements and releases. Since our development team is rather small and the nature of Lunar-Linux’s moonbase of rolling updates without the need of a full release, new articles on the site had been few and far between. To liven things up, we decided it would be nice to have developers or users contribute articles by turning our main site into a “planet” of sorts.

As you will see (or already have), posts will be aggregated from other blogs belonging to our developers and perhaps a few users. The first of which is Jean Bruenn, also known as “wdp” in our IRC channel who is a long-time contributor, mirror provider, and now one of our developers. Jean’s personal blog is where most of his posts originate, along side his own musings.

We are aware of some known issues about the migration. We weren’t able to bring over comments, but there were not many to begin with, so this should not be too much of a concern. Some articles or pages may have formatting issues, but we should be cleaning those up soon. Also of note, some new redirects have been made, some removed, others changed. Should you spot any issues that need our attention, please submit a bug report (yes, the website is on there) and let us know.

Lastly, if you would like your very own blog here, drop us a line on the mailing lists, or via IRC.

However, we still need a solution to handle renamed modules better than we do currently, so I came up with the following snippets:

Patch 1 to /var/lib/lunar/functions/modules.lunar

368a369,381
> # function : module_superseeded
> # usage    : module_superseeded $MODULE
> # purpose  : checks if $MODULE is superseeded by another MODULE
> # return   : returns 0 if superseeded, 255 else
> module_superseeded() {
>   if run_details $1 &>/dev/null ; then
>     if [ ! -z "$SUPERSEEDED" ]; then
>       return 0;
>     fi
>   fi
>   return 255
> }
>
546a560,572
>       fi
>     else
>       # run_details $MODULE worked; thus the module is installed
>       # let's check if it got renamed.
>       if module_superseeded $MODULE ; then
>         message
> "${MODULE_COLOR}$MODULE${DEFAULT_COLOR}${MESSAGE_COLOR} got renamed to
> ${MODULE_COLOR}$SUPERSEEDED${DEFAULT_COLOR}";
>         if query "Do you want to switch to
> ${MODULE_COLOR}$SUPERSEEDED${DEFAULT_COLOR}${MESSAGE_COLOR} ?" y ; then
>           lrm $MODULE
>           lin $SUPERSEEDED
>           continue
>         else
>           message
> "${MODULE_COLOR}$MODULE${DEFAULT_COLOR}${MESSAGE_COLOR} is kept and can
> be renamed manually later${DEFAULT_COLOR}";
>         fi

Patch 2 to /sbin/lin

142a143,146
>           elif module_superseeded $MODULE ; then
>             error_message
> "${LRM_COLOR}Notice:${DEFAULT_COLOR}${MESSAGE_COLOR} The module is
> superseeded by
> ${MODULE_COLOR}$SUPERSEEDED${DEFAULT_COLOR}${MESSAGE_COLOR}";
>             error_message "please lin that one
> instead.${DEFAULT_COLOR}"
>             continue

Remember, the goal was to keep it very simple. The above code changes the way, developers have to handle modules (if they rename them) a little bit. They have to set the SUPERSEEDED flag in DETAILS file to get it working. The devs I talked to have been okay with this approach.

Example:
renamed gcc to wdp-rulez

[lunar] root@lunar /var/lib/lunar/moonbase/compilers # purge_modules
+ Discovering modules that were removed from moonbase
flash-plugin-10 was removed from /var/lib/lunar/moonbase
flash-plugin-10: Do you want to remove flash-plugin-10 ? [y] n
flash-plugin-10 is kept and can be removed manually later
gcc got renamed to wdp-rulez
gcc: Do you want to switch to wdp-rulez ? [y] n
gcc is kept and can be renamed manually later
[lunar] root@lunar /var/lib/lunar/moonbase/compilers #

For devs handling of modules changes to this:

The proper way to handle renamed modules is now:

1. Copy the module to it’s new place and rename the copy.
   e.g: cp -rva compilers/gcc compilers/newgcc
2. Move the old one to zdeprecated
   e.g: mv compilers/gcc zdeprecated/gcc
3. Remove everything _except_ DETAILS from zdeprecated/gcc/*
4. Add: SUPERSEEDED=newgcc to DETAILS
   e.g: echo “SUPERSEEDED=newgcc” >> zdeprecated/gcc/DETAILS

El_Angelo suggested that we make sure that people can’t lin/get
the new module if they “lin” the old one.

Example:

root@lunar ~ # lin gcc
+ Spawning download manager
+ download queue:   gcc
Notice: The module is superseeded by  wdp-rulez
please lin that one instead.
root@lunar ~ #

That way we’re making sure, that nobody installs a package which
is superseeded by another.

Let’s see whether this will go into Lunar

CONFIG_DEVTMPFS=y
CONFIG_CGROUPS=y
CONFIG_AUTOFS4_FS=[y|m]
CONFIG_IPV6=[y|m]
CONFIG_RTC_DRV_CMOS=y
CONFIG_FANOTIFY=y
CONFIG_AUDIT=y

AUDIT is not a requirenment. IPV6 is not a requirenment i.e. it’s recommended to have them, not needed though.

Now we can continue by installing systemd using the following command:

lin systemd

The installer will ask you, whether you want sysv-compatibility, for now you should say “yes”, because of a few reasons. For example networking wouldn’t work correctly without it, as of writing this article. If it asks you whether you want cryptsetup keep in mind that this will install lvm2, which is not what I want, because I use device-mapper which would conflict with lvm2. So I’ll say “no”.

Suggestion: If compilation fails because of some error get it compiled by saying “no” to libnotify.

The next step in our list is removing everything not-needed from /etc/fstab. /etc/fstab should ONLY contain your real drives. That means, comment /dev/pts comment /proc comment /var* and tmp.. My fstab looks like this:

#UUID=83f6bd55-9686-46e2-b793-74a3e13332cd
/dev/sda1               /       ext4            defaults,noatime        0 1
#UUID=e66bbf62-783b-4276-b2f8-5bd05bbfd542
/dev/sda5               /home   ext4            defaults,relatime       0 2
/dev/sda2               swap    swap            defaults                0 0

There’s NOTHING ELSE in /etc/fstab. Now you need to add a symlink from /proc/self/mounts to /etc/mtab. If you don’t, you’ll get:

Aug  4 14:07:24 localhost kernel: systemd[1]: /etc/mtab is not a symlink or not pointing to /proc/self/mounts. This is not supported anymore. Please make sure to replace this file by a symlink to avoid incorrect or misleading mount(8) output.

so just do:

rm /etc/mtab && ln -s /proc/self/mounts /etc/mtab

The stuff below, is already integrated into lunar linux

That means, you don’t need to do this. This is just here, so you can see what I had to do when I initially tried to integrate it. However; Skip to “Migrating Services” below

You can try systemd now by adding init=/sbin/systemd to your bootloader. I hope it’s starting as fine for you as for me. However. Let’s configure systemd now. First we should set a nice hostname, in case you haven’t set one yet.

echo "yourhostname" > /etc/hostname

Now let’s configure console font and such stuff. Open /etc/vconsole.conf with your favorite editor and write something like this to it:

KEYMAP=de
FONT=sun12x22
FONT_MAP=8859-15_to_uni

As a little hint: You can check available keymaps in /usr/share/kbd/keymaps/. DE is german. You can check for available fonts in: /usr/share/kbd/consolefonts. I love this font. And the font maps are located in: /usr/share/kbd/consoletrans. 8859-15 is euro charset.

You can add the following to /etc/os-release:

NAME=Lunarlinux
ID=lunar
PRETTY_NAME=Lunar Linux
ANSI_COLOR=1;34

Usually you shouldn’t touch this, but as it doesn’t exist yet… Now add the locales:

LANG=en_US.utf8
LC_COLLATE=C

to /etc/locale.conf – I dislike german locales.. if you like them, replace en_US with de_DE. Basically this was it. You could play around with module blacklisting and loading now but that’ll be explained by me another time.

Migrating services

I hope you’re willing to improve Lunar Linux also – Issue:

systemctl

and scroll down. You’ll see some lines with “failed”. Please report them or try to find out, why they failed, try to fix them and let us know on IRC or by mail. Also take a look at those lines containing SYSV. Those are the compatibility-sysv-services which needs to be replaced i.e. we need to write a service for them. I’ll show you one example. You can help us to improve Lunar by mailing such services to us or by giving them to us on irc (using pastebin or something). Anyway. Let’s imagine we want to write a service for “slim” my favorite graphical login manager. First let’s go into /etc/systemd/system as it’s a system-service and not a user-service. Now open “slim.service” in your favorite text editor.

[Unit]
Description=slim - simple login manager
Requires=dev-tty7.device
After=dev-tty7.device systemd-user-sessions.service

[Service]
ExecStart=/usr/bin/slim -nodaemon
Restart=always
StandardOutput=syslog

[Install]
WantedBy=graphical.target

I guess, the stuff is pretty self-explaining. Now issue:

systemctl enable slim.service

And you’re done. That was all about it.

Connman

Currently we’re using some bash scripts and lnet to setup networking. Using connman this can be simplified a lot. El_Angelo and me are working on some GUI for connman right now, will take some time till it’s in moonbase. However, if you got a dhcp router connman should easily configure your wired network automatically – wireless lan should work also. If you have any trouble with connman ask us on IRC and we’ll try to help.

lin -cr connman
mv /etc/init.d/network /etc/init.d/network.old
/etc/init.d/network.old stop
systemctl enable connman.service
systemctl start connman.service

This will install connman, make a backup of your network init script i.e. if networking fails with connman just do /etc/init.d/network.old start, enable the service and start connman.

Check if /etc/resolv.conf is correct, check if ifconfig is correct, check if internet connectivity/network works. If yes – Try a reboot and check if your network comes up automatically.

For wlan, unpack the source (/var/spool/lunar/connman*) to /usr/src, switch into that directory, and there switch into the directory “test” within that folder, you’ll find a few test-tools which are handy to configure connman:

./list-services

find out your service (the part after /service/ starting with wifi_)

./test-connman enable service

(replace service with wifi_yourservice)

to enable the stuff. Now you can set the Passphrase:

./test-connman passphrase service the-phrase

(replace service again, and put in your phrase at the-phrase)

and you can try to enable autoconnect:

./test-connman passphrase service

hint #1
dveatch reported that he dislikes iptables and that connman moaned because of some xtable thingy. A simple lin of iptables solved that issue for him.

hint #2
we did some work on the wpa_supplicant module, it might well be that you have to relin it, so that it uses the new dbus interface and so that it has the nl80bla-driver enabled. even if you just use wired networking, connman might moan that it wants wpa_supplicant – So just install it using lin -cr wpa_supplicant.

Switching to non-sysv-compat systemd

Basically you just had to do the following: install systemd. boot with systemd. Migrate your services to systemd (at least the important ones), get connman working (thus our old network init script is not used anymore and your network works without sysvinit), reboot and check for any errors, solve them if possible and if there are any, recompile systemd without sysv-compatibility, reboot – done.

sqlite

time sqlite3 cache.db " select t1.id, t1.name, t2.category from lunar_modules t1 join lunar_modules_categories t2 on t1.category_id = t2.id where t2.category = 'shells';"

real    0m0.010s

lunar’s tools

time lvu section shells

real    0m0.093s

Looks like sqlite is 9 times faster than the lunar-tools. That’s not bad at all. However, these are just some pseudo numbers, we’re working on a patch so that everyone can test that. There’s a todo list with this db:

1. Implement dependency tracking
2. Write functions to do usual tasks (lvu depends, lvu where, lvu search, lvu website, lvu depends, get_all_modules, lvu expired, etc)
3. Write an entry in lunar features where users can enable this cached db optionally
4. Enhance the lunar tools (lvu, lin, lunar, etc) to use this database and the file-operations as fallback

The good thing about this solution is: It’s just a starting point. We can enhance this as much as we want. While just using it to improve some simple things now, we can use it also for complex things in the future. Also: Any changes made is non intrusive to the old behavior, they would co-exist and sqlite would be optional if implemented.