post

Things to improve Ubuntu 16.04 on Dell XPS 13 (9630)

Share on FacebookTweet about this on TwitterShare on Google+Share on LinkedInPin on PinterestShare on Reddit

Update 2017-04-30:

Since my laptop became more or less impossible to use with the WD15 dock and multiple external monitors, I had to continue looking for a solution. First I spent the better part of Saturday to try to create a Windows To Go installation on a USB stick in order to run the WD15 firmware upgrade (that only works on Windows). After several failed attempts I looked elsewhere for a solution. Turns out that for any 16.04 LTS installation, kernel upgrades are held back, so while a fresh install of 16.04.2 from CD would give you a Linux 4.8 kernel, systems like mine that had 16.04 from start, would still have a 4.4 kernel, the reason is that pretty much the only reason for upgrading the kernel these days is to get better hardware support  System that works fine with 4.4 will not get any benefits from an upgrade. But my system wasn’t working fine and the releases between 4.4 and 4.8, there has been a lot of Dell specific improvements and a bunch of usb-c improvements that could potentially make things better.

First, make sure you have all relevant upgrades so that your current system is in fact a 16.04.2.

erik@erik-xps ~ $ cat /etc/lsb-release
 DISTRIB_ID=Ubuntu
 DISTRIB_RELEASE=16.04
 DISTRIB_CODENAME=xenial
 DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS"

If your system still is 16.04.1, run the software updater manually and install everything it suggests.

When you have all available upgrades, check that your kernel is in fact held back with the following command:

erik@erik-xps ~ $ uname -r
 4.4.0-75-generic

If your kernel version is lower than 4.8 and you are experiencing hardware related issues (like my problems with the WD15), you may very well benefit from a kernel upgrade. The easiest way to do that is to enable something called the hardware enablement stack that instructs your system to install the latest kernels and X version. A warning might be in place, once enabled your system is basically going to get new kernels every time Ubuntu has a new point-release. So whenever 16.04.3 is released your kernel is most likely going to be upgraded again. Getting a new kernel every now and then might be too risky for some systems. Your mileage may vary. My reasoning is that since the XPS 13 9630 is still very new hardware (in Linux terms), new kernels will gradually make it better and better so I accept the small risk that some kernel along the line could potentially mess things up.

To get the hardware enablement stack, I issued the following:

erik@erik-xps ~ $ sudo apt install --install-recommends xserver-xorg-hwe-16.04

A few minutes and a reboot later, my laptop was working better with the WD15 than it did before the devestating 4.4.0.75 upgrade that started this whole mess.

The downside right now seem to be that I get no sound at all from the WD15 now. Neither the audio jack at the front or back seem to be working. Hopefully something that can be resolved but compared to no support for external monitors, a minor issue.

Power cycling the laptop AND the WD15 fixed the audio issue.

 

Update 2017-04-28:

After getting automatically upgraded to the latest Kernel (4.4.0.75) this week I had a proper sh*t storm of issues with my external monitors for two days. Sometimes they wouldn’t switch on after a cold boot. Sometimes I see the Ubuntu login screen on the external monitors for a second or two before they switch off. Sometimes the external keyboard and mouse wouldn’t work after boot.  Even the screensaver would start messing with the external monitors and keyboard, everything was basically random but with a strong bias towards not working at all.

I gradually tracked it down to being an issue with the WD15 dock and its usb-c connection. Some searching around led me to this thread on the Dell Community forums that made me realize I’m running an outdated BIOS, I had version 1.0.7 but the current version is 1.2.3 . I downloaded and updated the the new bios and after 1-2 hours of running, a lot of dock/usb-c related issues seems to be fixed. I recommend every user to use an updated BIOS, but if you’re using an usb-c dock with your Dell XPS, I’d say this BIOS update is mandatory.

New update: I spoke too soon, most of the issues remain the same.

New daily driver

This week my new daily driver arrived. A brand new Kaby Lake Dell XPS 13 Developers Edition (late 2016) that comes with Ubuntu pre-installed. It’s the first time in good number of years that I’m working with latest gen hardware and it’s also the first time I try a laptop with manufacturer support for Ubuntu. Interesting indeed. I also purchased a Dell WD15 dock to hook it up to my 2 22″ screens in the home office.

Apart from a few “paper cuts”, I just love this laptop. It’s the first time ever that I own a laptop with more than 6 hours of (realistic) battery life out of the box. The screen is gorgeous and the backlit keyboard feels really comfortable. If you’re looking for a high end laptop to run Linux, I highly recommend this one.

But as I said above, there are a few annoying things that needs to be improved. This blog post is my way of documenting the changes I’ve made so far and it’s very likely that I keep expanding this post as I discover and hopefully fix issues.

Touchpad

This laptop comes with a really nice trackpad. But when the computer first boots, it will have two separate trackpad drivers active. This makes synclient (the software that controls trackpad configuration) all confused and attach to the wrong driver. The trackpad will mostly work, but it’s not going to be possible to disable the trackpad while typing. This in turn means that if you enable “tap to click” on the trackpad, you will accidentally move the cursor around by “tapping” the trackpad with the palm of your hand driving you insane very quickly.

The solution is a two step process:

Install touchpad-indicator

touchpad-indicator is a utiility that sits in the upper right hand indicator area in Unity. This util gives you some additional configuration settings for the touchpad, “disable touchpad on typing” being the important one for me.

Touchpad indicator

Touchpad indiator UI

To install it:

$ sudo add-apt-repository ppa:atareao/atareao
$ sudo apt-get update
$ sudo apt-get install touchpad-indicator

After installing it, I had to start it manually once, and then tell it to autostart on it’s General Options tab.

$ /opt/extras.ubuntu.com/touchpad-indicator/bin/touchpad-indicatDisable the unneeded touchpad driver

Before touchpad-indicator can work, I also needed to disable the unnedded touchpad driver, the third answer in this thread explains how it’s done: https://ubuntuforums.org/showthread.php?t=2316240

Edit: Another thing that might improve the touchpad is to enable PalmDetect, I haven’t played around with it enough to know if it matters or not, but I have to add a line int a X11 config file to enable it:

$ sudo nano /usr/share/X11/xorg.conf.d/50-synaptics.conf

And then after line 13 i added ‘Option “PalmDetect” “1”‘

edit 50-synaptics.conf

 

 

WD15 Dock

There’s a lot to say about the Dell WD15 dock. For the most part, it works as expected but there are some annoying buts that goes with it. From researching online I realized that with the Linux kernel that comes with stock Ubuntu 16.04 a lot just works and for that I’m thankful. The poor customers that tried to make this dock work with previous versions of Ubuntu have suffered much more than I have. There are a few things that doesn’t work though.

Audio

The WD15 has a 3.5 mm loudspeaker jack on the back that doesn’t work and an similar 3.5 mm headphone jack on the front that does. Not a huge deal for me, I still get decent quality sound to my external speakers, but the installation could have been prettier:WD15 dock

The other annoying thing with the dock is that I have a ton of trouble making it understand when to enable the external monitors, when to wake up from suspend and what resolution to use. I’ve had similar issues with other docks (HP) in the past. I don’t have a solution for it, I guess I just slowly adjust to a lot of rebooting and manual screen resolution management.

The super key

One of the most odd things with this laptop is that the pre-installed Ubuntu 16.04 comes with a Dell specific package named dell-super-key. This package seem to do just one single thing: disable the super key. If you’re the least bit familiar with Ubuntu, you know that the super key used a lot so exactly what the product developers at Dell was thinking is a mystery to me. Why?

Anyway, it’s easy to fix. Just uninstall the dell-super-key package and you’re good to go.

$ sudo apt-get remove dell-super-key

Conflicting keyboard mapping

I’m not sure if this is specific to the Dell installation of Ubuntu or not but I haven’t had this issue on any other laptops, including my last HP that was also running 16.04. I work a lot with different works paces and I use Ctrl+Alt+Up/Down to move between them. On this one, there was a conflict in mapping so that the Ctrl+Alt+Up combo was also mapped to “Maximize horizontally”. Whenever I had focus on a window that could be maximized, Ctrl+Alt+Up would maximize that window instead of taking me to the work space I wanted.

Searching around in the standard places for where to fix this turned up nothing. I disabled the maximize action in every place I could think of; System Settings -> Keyboard -> Shortcuts as well as using the dconf-editor. Turned out to be the Compiz plugin “Grid” that caused the problem. I solved it by simply disabling these keyboard mappings from Grid.

First, install the Compiz settings tool:

$ sudo apt-get install compizconfig-settings-manager

When install, launch it and search for the Grid plugin:

Compiz Config Settings Grid

Then in the Grid settings, click to disable and then re-enable the Grid plugin, it will detect the keyboard mapping conflicts and ask you how to resolve them. I told Grid to not set any of the keyboard shortcuts that conflicted with the “Desktop wall” plugin. That way I can keep some of the Grid features I like, such as maximizing a window by dragging it to the top of the screen:

Resolving conflicts in compiz settings

 

Conclusion

Compared to 10 yeas ago when I first started using Linux as my primary OS, the tweaks needed to make this laptop work as I want it are minimal. Linux and Ubuntu have come a long long way since then and it’s world of difference.

It would be easy to point a a finger at Dell for shipping a laptop with these issues, but I think that would be very unfair, instead I applaud them for sticking to their Developer Edition program. Sure, the super key thing is weird and perhaps they could have solved the touchpad thing better, but those are solvable. I prefer Dell to keep assembling great hardware, after all, there’s a great community of Linux users around to get the last few issues resolved.

If you have any questions or if you’ve found another Ubuntu and Dell XPS related issue, please use the comment field below.

 

Gmail and Google Apps mail migration

Share on FacebookTweet about this on TwitterShare on Google+Share on LinkedInPin on PinterestShare on Reddit

I’ve been a long time Google Apps user, I think it’s a perfect solution for a smaller company like mine. In fact, it’s a perfect solution for bigger companies as well. Right now, I’m working with a client that wants to consolidate 10 individual Google Apps domains into one single account that handles all the domains. About half of the 15 users have accounts in all the other domains, the other half have accounts in 2-3 of them. Even if this is not the most typical kind of work I’m doing, this assignment brings some welcome new challenges into my current work.

Migration tool requirements

Anyway. One of the client requirements is naturally that all existing email is migrated into the new Google Apps account. Easy peasy right? Well, it turns out that migrating email to and from Gmail (and therefore Google Apps) is quite a challenge due to a number of reasons. Of course, there are plenty of advice on the Internet, but none of the proposed solutions would cover all my needs which are:

  1. The tool must be able to handle XOAUTH on the source.
  2. The tool must be able to handle XOAUTH on the target.
  3. The tool must be able to handle Gmail’s rather special model of treating folders like labels.
  4. The tool must be scriptable
  5. The tool must be able to handle delta changes, running the script a second time should not create duplicates on the target email account.

I’ve looked at plenty of alternatives but the only tools I found that could potentially do the work for me came at a too high cost. The cloud based tools that exist typically charge per user account, that would have been fine if there was a 1-1 mapping between users and accounts. But in my scenario, each user have on average 7 accounts and I had given my client a fixed price for the entire job. So, even if I had loved to try, I can’t afford to lose money on a job.

The two most problematic requirements was to handle xoauth on both ends and to handle the gmail folder/label magic. The two official tools from Google, the migration API and the Google migration tool for Exchange failed. The API only gives you write access, so it’s not possible to get email OUT of a Gmail account using it. The Google Exchange Migration tool assumes that the source server is something other than Gmail and requires you to know the username / password for all source accounts.

A solution… almost

Enter imapsync. Imapsync used to be a free open source tool that is now under a commercial license. But for a mere EUR 50, I bought access to the source code (in perl). Imapsync is able to handle XOAUTH on both source and destination, it’s scriptable and it’s able to use MessageId to keep a kind of state. Running imapsync twice with the same parameters will not duplicate the amount of emails on the target server, more on that later.

The one problem I had with imapsync was the folder vs label management. The problem that most people know of is that Gmail doesn’t really use folder, it uses labels. Even if it’s similar in a lot of cases, there are differences. What I learned is that there’s another issue regarding the concept of subfolders or nested labels. An example

  • Via IMAP, create a folder  named foo. => Gmail creates a root level label “foo”. 
  • Via IMAP, create a folder named foo/bar => Gmail creates the label “bar” nested under the label “foo”.
  • Via IMAP, create a folder named “fuu/bar” => Gmail create the root level label “fuu/bar”.

See the difference? In the last example, you’d perhaps thought that Gmail would create a root level label “fuu” and then a nested label “bar” under it. But nope, Gmail will happily create a label containing the actual IMAP label separator character. Bummer. So the end result is that if you transfer email with imapsync out of the box, you will get a flat structure of really long label names. And that flat list that can grow to be quite long if your’re actively using nested labels. And you don’t want that.

I was pondering a whole lot of various solutions to this problem. I actually got to the point where I tried to migrate the source account to a local IMAP account on my own machine, manipulate the Maildir directly on disk to insert dummy email in strategic places and then migrate to the target account. It worked, but it also introduced a whole new set of moving parts.

The final solution (thanks Dennis)

dennis_filter

It took a long sunday walk with the dog before I realized that the proper solution would be to work with the imapsync source to fix folder creation. As I described above, the cause of the folder / label problem is that Gmail treats things differently depending on the order of folder creation. So, after the initial shock of seeing 5000 lines of Perl code (I don’t consider Perl to be part of my standard toolbox) I got to work and built me a patch. With the patch in place, the folder creation now works as I’d expected it in the first place. The one downside to this solution is that it won’t be able to see the difference between a label on the source Gmail account that actually contains a / (forward slash).

The other thing I with this patch is that it don’t have a switch to tell imapsync if you want the different folder creation behavior or not. I guess that’s needed before I submit it back to the maintainer.

Anyway, this patch assumes that you have imapsync 1.542 even if it’s likely to work well with other versions as well. If you have another version of imapsync and want to work with Gmail migrations, consider upgrading anyway since only 1.542 supports xoauth. On line 2312, replace the existing create_folders function with this modified version:

sub create_folder {
	my( $imap2, $h2_fold, $h1_fold ) = @_ ;
        my(@parts, $parent);

	print "Creating folder [$h2_fold] on host2\n";
        if ( ( 'INBOX' eq uc( $h2_fold) )
         and ( $imap2->exists( $h2_fold ) ) ) {
                print "Folder [$h2_fold] already exists\n" ;
                return( 1 ) ;
        }

        @parts = split($h2_sep, $h2_fold );
        pop( @parts );
        $parent = join($h2_sep, @parts );
        $parent =~ s/^\s+|\s+$//g ;
        if(($parent ne "") and !$imap2->exists( $parent )) {
        	create_folder( $imap2 , $parent , $h1_fold);
        }

	if ( ! $dry ){
		if ( ! $imap2->create( $h2_fold ) ) {
			print( "Couldn't create folder [$h2_fold] from [$h1_fold]: ",
			$imap2->LastError(  ), "\n" );
			$nb_errors++;
                        # success if folder exists ("already exists" error)
                        return( 1 ) if $imap2->exists( $h2_fold ) ;
                        # failure since create failed
			return( 0 );
		}else{
			#create succeeded
			return( 1 );
		}
	}else{
		# dry mode, no folder so many imap will fail, assuming failure
		return( 0 );
	}
}

 

 

Net ID on Ubuntu 12.10

Share on FacebookTweet about this on TwitterShare on Google+Share on LinkedInPin on PinterestShare on Reddit

I don’t know if it’s because of my desktop environment (Ubuntu 12.10, Gnome Shell 3.6) is so darn special or because Firefox 17 disabled a certain feature. But anyway, when I tried installing Net ID from SecMaker, better known as Telia e-legitimation, it would just fail. But I was able to find a workaround.

My symptoms: I followed the instructions from Telia / SecMaker and it semed OK. To finish the actual installation part, you’re supposed to browse to /etc/iid/pkcs11.html to let Firefox install Net ID as a local security device. However, that part simply doesn’t work. I got the message:

 

Installation status for Net iD pkcs#11 module is unknown, use buttons below to install or uninstall the module.

But the buttons Install / Uninstall didn’t actually do anything.

  1. The problem: The whole point of surfing to /etc/iid/pksc11.html is so that a javascript can install Net ID as security device. Turns out that Firefox 17 have disabled access to the internal Firefox object that can manipulate the list of security devices unless the code executes inside a proper Firefox extension. In this case, the code executes inside a normal webpage so it simply won’t work.

The solution: It turns out that there’s another method to install Net ID as a security device and that  is to do it manually. This is how.

1. Open Edit -> Preferences -> Advanced.

2. Select the Encryption tab

3. Click Security Devices. You should see something like the below (except that you won’t see “Net ID” in the tree view on the left hand side.Screenshot from 2012-12-18 10:08:57

4. Click the Load button on the right hand side. Fill in the details. You want to name the module “Net ID” and give the path to the .so file to /usr/lib/libiid11p.so

Screenshot from 2012-12-18 10:14:44

5. Click OK. You should now see Net ID in the tree view on the left hand side and that’s our goal here, so, hooray!!!

Now, you have added the Net ID software to your computer and made it available in Firefox, so you installation can continue as described in the instructions from Teliea (or whatever ID provider you have).

Good luck!