ZenLinux Blog

I returned to China during a three country East-Asia trip this Spring, and thought I’d share some more notes on being able to work remotely while in China. Some things worth sharing have changed since my last blog post on the topic in 2016.

One thing that’s interesting about the Great Firewall (GFW) is that China uses it for censorship of its mainland residents, but doesn’t do so for residents of Hong Kong, even though many of the same Chinese telecoms offer services there. I’ve heard that if you buy a prepaid sim card in Hong Kong, and use it in mainland China, your cell data service is not blocked by the GFW. So I was going to buy a prepaid Hong Kong sim before my trip until I learned that you can also buy sim cards from foreign providers that can work at 4G speeds in roaming mode in multiple countries. As I was traveling to Japan, Korea, and China, I started looking for a single sim solution that would work in all three countries.

What I found (and which worked flawlessly) was the AIS sim2fly prepaid sim card, which you can buy on Amazon. AIS is one of the biggest telecom companies in Thailand, and they claimed that these cards worked at full 4G speeds in roaming mode in several Asian countries, and that the service was not blocked by the GFW when used in China. They offer an 8-day prepaid sim with 6 GB of data (tethering supported), which was more than enough data for me. I ended up buying a few of these, so that after 8 days I simply popped in a new sim card and I was good to go for another 8 days. On top of that, using these sim cards was cheaper than if I were to have bought separate sim cards for Japan, Korea, and China. I’d highly recommend the AIS sim2fly prepaid sims for these kinds of trips.

As for the times I had to use a VPN over wifi, I did some more research and learned that Astrill is still a reliable provider. As of late, ExpressVPN seems to have become mostly unusable in China based on my research, though I didn’t try to use it personally during this trip. Anytime I had to use Astrill (which I typically used in their Wireguard mode), my speeds were extremely slow (1-2 Mbps) compared to what I’d get tethered to my smartphone AIS connection. Also my VPN would disconnect at random times – sometimes it would work reliably for a half-hour or more, and other times it would disconnect every few minutes. So my advice is if you don’t need to stream a lot of data, it would be far more convenient to rely entirely on smartphone tethering for your internet needs, assuming you’ve got good cell data coverage in the area you’ll be visiting. For any major city in China, this will be a non-issue.

An unrelated observation I had while in Beijing was that none of the locals used cash – everyone paid for things using WeChat Pay.
Unfortunately, you can’t link a foreign debit card to your WeChat Pay account – it only works with cards issued by Chinese banks.
So I was often the annoying foreigner who paid for things with cash. At one bakery, they even refused to break a 100 RMB note (worth around $14 USD) because they didn’t have enough change in their register. Being able to use WeChat pay unlocks a lot of other conveniences you can use in China, such as Didi (their equivalent of Lyft/Uber) rideshare payments, bike rentals, etc. So if you’re going to be in China for a long time (e.g, over a month), it may be worth the effort to open a Chinese bank account and keep a small amount of money in it for use with WeChat Pay.

Last month I visited Beijing, China and had to work remotely during my trip. At work we rely on a number of Google services, so I needed a reliable way to circumvent the Great Firewall of China (GFW). After doing a decent amount of research, I learned that just running a SOCKS proxy via SSH is likely to run into problems, so I used a couple of commercial VPNs, as well as a private Shadowsocks server I had set up on various ports of a Digital Ocean droplet. The idea being to have a couple of fall-back methods to tunnel through the GFW in case my primary one stopped working. I thought it might be useful to report on what worked well, and what was most challenging about this.

Given that I’m a Linux user and needed solutions that were Linux-friendly, I settled on two highly recommended commercial VPNs – ExpressVPN and Astrill. I also sprung for the added “VIP” add-on to Astrill that gives you access to a few additional VPN endpoints that presumably have lower utilization. In summary, Astrill was the clear winner, especially with the VIP add-on. Though no matter which VPN service I was using, there was a lot of fiddling that had to be done to test the latency of different proxy endpoints. There wasn’t one I could just set and forget.

Finding usable wifi in Beijing is another story, and proved to be a frustrating problem. My local resident friend told me that the Chinese tend to use the internet for recreation rather than getting work done, so the vast majority of folks packed in coffee shops are streaming video to watch movies or TV shows. My own observations backed this up, and it was easy to notice this, as a sizable proportion of these folks don’t bother to use headphones when watching their entertainment (grumble). So I found the only times I had really solid wifi speeds were when I found a coffee shop that was mostly empty, and probably half the time I gave up on the wifi and just tethered to my phone’s data connection. For most of my work I was running remote builds over SSH, and I found my phone’s data connection was laggy in a more consistent way than when I tried to use wifi in a busy cafe.

Regarding SIM cards in China, I have some tips to share as well. I ended up buying a prepaid China Unicom SIM with 2 GB of data from Amazon before I left for my trip, which was incredibly convenient. The way this works is you buy the SIM online, they send it to you, and you have to activate it over email with the seller. Once the SIM is activated, the 90-day lifetime of the SIM doesn’t start until you actually begin to use it, so you can complete the activation well before your trip and then pop the SIM card into your phone once you land in China. I have no complaints about dealing with the seller LvyCom on Amazon and would definitely recommend them.

So how was ExpressVPN? Decent and reliable, but not especially fast. I found it helped significantly to change the connection type from “auto” to “udp”, but Astrill’s Openweb connection type still beat it when it came to speeds. But to set expectations – generally the speeds were still slow. My friend had an 80 Mbit home internet connection which I tested without the VPN enabled, but once I enabled a VPN, the best I could get from it was around 3-5 Mbit. This was generally only good enough to watch YouTube videos at 480p. My friend was quite surprised when I told him I always watch YouTube at home at 1080p resolution with no hiccups or delays.

Shadowsocks turned out to be the least reliable method of tunneling out of China, sometimes working well and sometimes not working at all. Since it’s a lot of extra effort to set up a Shadowsocks server compared to just using a commercial VPN, I don’t think it’s necessary unless you want to have that extra peace of mind.

Overall I was able to get work done while in China, but it was regularly a frustrating experience to deal with the lack of bandwidth and annoying latency on SSH terminal sessions. Oh, and bring good headphones if you plan to try to work from coffee shops!

Photos from my recent trip to Beijing can be found here. For news about the GFW and VPNs, I recommend greatfire.org.

If you’re in the Portland area and are curious to learn a few things about the Autotools, please feel free to attend the December meeting of the Portland Linux User Group at PSU. I’ll be giving a novice-friendly introduction to the Autotools.

Autoconf. Automake. Libtool. This trio of build configuration utilities (known as the Autotools) are used in a large majority of compiled software applications for Linux, but they remain a mystery to many of us.

Finally, Scott will offer some practical tips for understanding and fixing errors you may see when building an Autotools-based package. It’s sure to be a fun romp for the whole family.

When: 7-9pm Thursday, December 1, 2011
Where: Portland State University Engineering Building, room FAB 86-01 (this is in the basement). The building is on SW 4th Ave across from SW College Street.

chroots provide an extremely useful way of running multiple Linux distros on a single computer without having to run them within a full virtual machine environment. Why would this be an advantage? Well, in my case, I’m performing builds of hundreds of packages for the Yocto Project, and I want to maximize performance by avoiding having to go through a virtualization layer to access my hard disks.

My goal was to create a minimal environment which closely matched what we’re running on some of our autobuilders, which happens to be openSUSE 11.2. It’s easy to set up Debian-based distributions within a chroot by using the debootstrap command, but openSUSE doesn’t have anything analagous to this AFAIK.

It turns out it’s pretty simple to use zypper, openSUSE’s command-line package management utility, to bootstrap an installation into an arbitrary sysroot that you can then use to pull in additional packages over the network. The only caveat is you have to create your initial chroot on an existing openSUSE machine where you have root privileges. Here’s how.

The first thing you’ll need to do is set up the most basic openSUSE package repository config within your sysroot. To create this in the /data/opensuse-11.2 directory, you’d do the following:

mkdir /data/opensuse-11.2
zypper --root /data/opensuse-11.2/ ar http://ftp.osuosl.org/pub/opensuse/distribution/11.2/repo/oss/ repo-oss

Next, you’ll need a /dev/zero device file within your sysroot, since some packages have post-install scripts which make use of it. You can create it manually, but I prefer to simply copy it from /dev:

sudo mkdir /data/opensuse-11.2/dev
sudo cp -a /dev/zero /data/opensuse-11.2/dev/

Presumably you’ll want to move this chroot onto another computer as soon as possible, so here I only install a few needed packages to get started (rpm, zypper, wget, and vim). An editor can be handy in case you need to edit a configuration file (in my case, to configure system-wide proxy settings in /etc/sysconfig/proxy):

sudo zypper --root /data/opensuse-11.2/ install rpm zypper wget vim

At this point you can now tar up the chroot and copy it over to your destination machine of choice. Keep in mind you’ll still need root privileges to tar it up properly:

cd /data
sudo tar cvjf opensuse-11.2.tar.bz2 opensuse-11.2

I use schroot to manage my chroots and highly recommend this little-known utility. It allows you to work within a chroot’ed environment but still have access to your home directory (or any other directory, if you configure it as a bind mount).

After having moved my chroot to its final destination, I added additional zypper repositories and followed the Yocto Project Quick Start Guide to install required build dependencies.

zypper ar http://download.opensuse.org/update/11.2 repo-update
zypper ar http://download.opensuse.org/repositories/security/openSUSE_11.2/ security
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools:/1.7/openSUSE_11.2/ tools
zypper refresh
zypper install python m4 make ...

From that point on, I could fire up a schroot session anytime I needed to perform builds within a minimal openSUSE 11.2 envrionment with no need to deal with virtualization environments or reboot into another OS.

Last year marked a new first for Portland, OR – the birth of the Open Source Bridge technical conference. In a previous post I expressed a great deal of enthusiasm about how awesome last year’s event was. Those weren’t just kind words – I found myself moved enough by the incredible activism and community in the Portland tech scene to get involved for this year’s conference as a volunteer. That’s right, Open Source Bridge is back in 2010! June 1-4, to be exact.

This year’s event has an outstanding presentation lineup and will be held at the Mark Building of the Portland Art Museum. I had a chance to tour the venue with the OSB organizing crew and must say that the location is really unique, inspiring, and truly fitting for a conference of people who are working to improve the world through quality open source software projects. There will once again be a 24-hour hacker lounge (a major highlight from last year), this time on-site at the Mark Building.

One of the great things about OSB is that it’s a very diverse gathering of open source citizens, and offers a great opportunity to expand your horizons to learn about tools and platforms you may not have encountered before. I will also be giving a variation of my PLUG Advanced Topics talk on OpenEmbedded if embedded Linux systems pique your interest.

Check out the Open Source Bridge website to learn more and register. Trust me – it’s gonna rock your socks.

A big warning to the many users behind a proxy server who will be installing Ubuntu Lucid Lynx soon: wget isn’t going to work. You can read the details in this bug report on Launchpad.

Basically the problem is that if your proxy exclusion list ends with a comma character, wget can’t parse the $no_proxy environment variable and defaults to not using your proxy (usually configured in ~/.wgetrc or via the $http_proxy/$ftp_proxy environment variables). And the GNOME Network Proxy UI apparently leaves a trailing comma in there if you make changes to the ignored hosts list.

The quick workaround is to add the following to your ~/.bashrc:

export no_proxy=$(echo $no_proxy | sed 's/,$//')

I spent a couple of days figuring this out, so I hope it helps someone else. I don’t see any way the fix will be included in Lucid before it ships on Thursday, so people will need to use this workaround until then. It’s a shame because as an LTS release, a lot of enterprise users are going to run into problems right out of the gate.

SpamAssassin is one of those mission-critical services that I run on my mail server, and if you haven’t heard, there is a bug in SpamAssassin that has been marking legitimate messages (ham) as spam if the date of the email was 2010 or later. Now that it really is 2010, this is a serious problem. More details about the bug can be found here.

I’ve confirmed that my CentOS 5.4 install was vulnerable and I had to apply a workaround. There are a couple of ways to do this. You can either edit your local.cf file and disable the rule with the following line (on CentOS it’s in /etc/mail/spamassassin/local.cf):

score FH_DATE_PAST_20XX 0.0

Or you can enable the cron job to run sa-update nightly, which I would recommend. My CentOS system had the cron entry commented out in /etc/cron.d/sa-update, so I uncommented it.

If you’re running spamd on your system, don’t forget to restart the service for the new rules to be reloaded.

I feel the need to spread this ssh tip that saves me from quite a bit of typing on a daily basis. I learned about it from Elliott’s OS X Tips and Tricks post on the Carsonified blog.

Add the following to your ~/.ssh/config file:

Host *
ControlMaster auto
ControlPath ~/.ssh/master-%r@%h:%p

Now when you ssh into a host, subsequent connections to that host use the same TCP socket, and don’t require authentication. This will be the case for as long as that initial connection stays open, and it works for sftp as well as ssh.

I realize another way of avoiding typing your password all the time is to use ssh keys, but I happen to work on embedded systems that get rebuilt very frequently and which I’m not able set up an ssh key as part of the build process. This technique allows me to log into the system once per session and not have to type the password over and over again.