Index ¦ Archives ¦ RSS > Category: misc

Metric Won!

It turns out the metric system has completely won, but some just haven't realised it yet! If you are in the US, and need a definitive ruling on distance, weight etc then you'd think that somewhere there is the golden definition of an inch, a gallon, a pound and so on. There is. They are defined in terms of the metric system.

To quote at 8m45s in:

The most ridiculous thing about all of this? Every single one of these imperial measurements are legally defined by the metric system. America is already using the metric system, and most of the population is oblivious to is.

The imperial system wasn't even sensible. The US uses feet, but also uses surveying feet which are finally becoming one. US and UK gallons remain gratuitously different, so miles per gallons don't translate.

The final frontier are recipes randomly switching amongst volumes and weights with imperial units (quick: what is the weight difference between a fluid ounce and an ounce of water or butter?)

Category: misc – Tags: metric


A decade of hindsight

I wrote a bunch of stuff over the last 10 years. Now that we know what happened, this is my look back. Followup is most recent posts first, and then getting older.

History podcasts did well, and keep getting better.

I liked the Casio Smartwatch, but WearOS and its apps aren't getting much development. (It was particularly frustrating when even Google didn't bother to keep their apps working, and some third party ones just stopped working one day.) It turns out the Apple Watch is also 50m water resistant, will also run for 30 days in power reserve mode, and can also be charged without taking the watch off. Those were the important base features of the Casio to me. Apple watch also has a 1,000 nit display (ie sunlight readable), and you get microphone, speakers, and NFC. Plus the rectangular display is better for those like me who prefer digital watchfaces with lots of information. Even Google's apps work better. I switched.

I keep wishing Emacs well. Language servers have made development environments easier. In the end I did abandon Atom in favour of Visual Studio Code. While vscode doesn't have tramp mode, the remote development is good enough.

I was very wrong about Mario Kart 8. It is a lot of fun, and Nintendo fixed many of the Wii version issues. We try the Wii version again every now and then, and it seems less fun than we remembered.

I had to switch from Nikola to Pelican. The main feature of Pelican is a far slower development pace, and not sucking in lots of dependencies. Nikola went full tilt adding many features quickly, but that made it hard to run infrequently since everything would be a lot harder to update. Additionally every time I ran it, there was a blizzard of messages about deprecations and configuration changes.

Support is still a problem. It is still usually treated as a cost centre, with incentives to do as little as possible. It is easier to support smaller numbers of customers who have paid more for a product, but offering supoort to large numbers of people cheaply doesn't seem to be done by anyone.

SSL was fixed by Lets Encrypt.

RSS is still around, but not as mainstream as the days of Google Reader. I still use it.

Self driving cars are still just around the corner, while there is more evidence of just how bad human drivers are.

I still have trouble with voice recognition. Most of the services do get it mostly right now, but when they get it wrong it is very wrong. Any other humans in the room usually also burst into laughter, due to what the service did. For example I may ask for a temperature conversion, and instead the service will start reading out some obscure fact.

Category: misc


Exit Review: Python 2 (and some related thoughts)

Python 2 has come to an end. I ported the last of my personal scripts to Python 3 a few months ago.

Perhaps the greatest feature of Python 2 was that after the first few releases, it stayed stable. Code ran and worked. New releases didn't break anything. It was predictable. And existing Python 2 code won't break for a long time.

The end of Python 2 has led to the end of that stability, which isn't a bad thing. Python 3 is now competing across a broader ecosystem of languages and environments trying to improve developer and runtime efficiency. Great!

I did see a quote that Python is generally the second best solution to any problem. That is a good summary, and shows why Python is so useful when you need to solve many different problems. Ii is also my review of Python 2.

So let's have some musings ...

Python has had poor timing. The first Python release (1994) was when unicode was being developed, so the second major Python version (2000) had to bolt on unicode support. But if it had waited a few more years, then things could have been simpler by going straight to utf8 (see also PEP 0538).

Every language has been adding async with Python 3 (2008) increasing support with each minor release. However like most other languages, functions ended up coloured. This will end up solved, almost certainly by having the runtime automagically doing the right thing.

Python 3 made a big mistake with the 2to3 tool. It works exactly as described. But it had the unfortunate effect of maintainers keeping their code in Python 2, and using that to make releases that supported both Python 2 and 3. The counter-example is javascript where tools provide the most recent syntax and transpiling to support older versions. Hopefully future Python migration tools will follow the same pattern so that code can be maintained in the most recent release, and transpiled to support older versions. This should also be the case for using the C API.

THe CPython C API is quite nice for a C based object API. Even the internal objects use it. It followed the standard pattern of the time with an object (structure) pointer and methods taking it as a parameter. There are also macros for "optimised access". But this style makes changing underlying implementation details difficult, as alternate Python interpeter implementations have found out. If for example a handle based API was used instead, then it would have been slower due to an indirection, but allow easier changing of implementation details.

Another mistake was not namespacing the third party package repository PyPI. Others have made the same mistake. For example when SourceForge was a thing, they did not use namespacing so the urls were sf.net/projectname - which then led to issues over who legitimately owned projectname. Github added namespaces so the urls are github.com/user/projectname. (user can also be an organization.) This means the same projectname can exist many times over. That makes forking really easy, and is perhaps one of the most important software freedoms.

Using NPM as an example, this is the only package that can be named database. It hasn't been updated in 6 years. On PyPI this is apsw and hasn't been updated in 5 years. (I am the apsw author updating it about quarterly but not the publisher on PyPI for reasons.) Go does use namespacing. A single namespace prevents forks (under the same name) and also makes name squatting very easy. Hopefully Python will figure out a nice solution.

Category: misc – Tags: exit review, python


Recommended: History of podcasts

I'm a fan of podcasts and especially longer form history podcasts. I've found that "History of" podcasts that cover various empires and locations seem to be rather good. The History of Rome podcast is a very good example, with many others following that format and principles. The format allows the shows to adapt over time, include listener feedback, and do experiments which often work well.

If you can't get enough, then Hardcore History has many good episodes and stories.

And at the meta level, there is a History of *History of podcast* podcasts

Category: misc – Tags: recommendation


My Casio Smartwatch WSD-F30 experience

Summary

The manual (pdf) is comprehensive and describes the non-WearOS functionality well. r/WearOS covers the WearOS side - check the sidebar too. It is also worth noting that current watches tend to use identical hardware (same qualcomm chipset, same screen resolution, same RAM, same storage etc) although extras like microphones, speakers, NFC differ.

Starting point

I've used Casio digital watches for as long as I can remember. Because they are water resistant, the watch can go anywhere I do, and I never take them off. My favourite over the last decade has been the Solar Atomic models. Solar means I never need to change the battery, and "atomic" means picking up the radio time signals that came from an atomic clock.

Smartwatch?

Watches provide two conveniences for me - it is always there, and I can look at it very quickly. Phones are in chargers, pockets, etc and take longer to extract and navigate to what you wanted to see.

Needing to be familiar with smartwatches, and to do development work I naturally picked the Casio offering which is upper mid-range in pricing.

First Time User Experience (software)

The FTUE is terrible. Android Wear WearOS watches are not mature yet, and require a lot of compromise to keep within available battery, cpu and software functionality. It feels a lot like being given a decade old phone and told to make it work now.

Simultaneously the watch will be doing system updates, installing or updating apps, and have some tutorial overlay you can't just dismiss. All the while you are learning the compromises you'll have to make.

To be clear - it is sluggish. There will be 5 seconds between taps and resulting actions. The screen will go black for several seconds while apps launch. You are never certain if touches or button presses registered, and often end up doubling them which makes things worse. I also found the onscreen keyboard useless since I could never touch the right spot.

Things do settle down over time, but that sluggishness still remains some of the time. What helped me the most was to enable developer options and turn on "Show Taps". That confirms a tap was registered and shows where is was, helping with feedback and making the keyboard more useful.

Charging

Charging is done with a magnetic attached cable. The box came with a small USB power brick, and the USB to round magnet charging cable. I have never used the supplied power brick, and have had no problem connecting to any USB power source. I also bought a third party USB C to magnet off Amazon, and use it the most of the time. In short the watch is not fussy about charging.

When sitting at my desk, the cable will stay in place providing there isn't too much unsupported cable length, so that is the main way I charge the watch.

Watch Display

  • A monochrome digital time display, easily readable in sunlight and difficult to read in low light. Uses a lot less power than the colour display. You can run in this mode for 30 days with WearOS turned off. When WearOS is running then only Casio apps can write to this screen (other apps just have the standard time display)
  • Ambient mode colour display (lowest brightness). Unreadable in direct or indirect sunlight. This is used when idle with power consumption based on how many pixels are not black.
  • Colour display which uses lots of power, is readable in indirect sunlight and generally impossible to read in direct sunlight.

If you have the full colour display on and are interacting with apps, a full battery will be drained in about an hour. Consequently much use of the watch is setting the display mode you want to trade off power consumption, readability, and response time.

You can have the display activated by touch, button press, and rotating your wrist. My experience of wrist activation is that it rarely works when you want it too, and often activates when you don't. Because it activates full brightness, the battery can be very quickly drained.

Thoughts

WearOS is a lot less mature than expected. It is unclear if Google is losing interest.

Most watch faces try to be pretty and based on analog hands. It is difficult to find dense digital displays.

The Casio apps do work well. I'm glad Casio used WearOS instead of doing their own operating system with limited apps etc. However the result including their gshock style case seems pricey. A few more years of new models should improve this.

Ultimately you figure out how to get the watch to work for you, requiring more administration than a non-smartwatch. For me the benefits outweigh the hassle. I use Theater Mode from quick settings to have the time showing most of the time.

Category: misc – Tags: review


On defaults

I've been wondering what best practise for handling defaults is. In software there are generally 3 values: zero, one, or many. As a consequence developers often pick a sensible number for "many", and allow configuration to change it.

Eventually defaults permeate the code, settings, user interfaces, product documentation, user forums, and search engine results. It spreads not from a single source of truth that tracks and propagates changes, but by being arbitrarily copied between systems.

As time passes, the default values need to change due to circumstances and experience. New features make existing values need refinement, while new interactions complicate matters.

The usual solution is to bump the major version and have humans, code, and documentation deal with changes. The effort of doing major version upgrades especially all the setting changes is what makes so many of us resistant to do major version upgrades.

Starting software after a version upgrade is always a pain. Sometimes you are pleasantly surprised that it just works, but usually the logs are full of complaints about settings, things that previously worked no longer working and general yak shaving.

Postfix has a compatibility level to help defer the effort after a major version upgrade, but you are still on the hook for the upgrade changes.

An anti-pattern is software that generates an initial config file for you. It does have a very short path between default settings and the generated config file, usually including comments and explanations in that file. This is fantastic to start with.

But it causes problems in time. The settings, comments and explanations become wrong. Looking at a config file that is a few years old is an exercise in archaeology and contradictions, requiring consulting the file, warning/error messages, logs, wikis, and other documentation.

So far the best I have is to prefer more 'automatic' settings, and keep the number of settings to a minimum.

Category: misc


Exit review: Emacs

A shocking time has come - I've given up Emacs, after using it for 20 years. When interviewing developers, one of the questions I ask is about their favourite editor. I don't care what the answer is, but I do very much care about why it is. An editor is a fundamental part of developer productivity, so I want to hear about the candidate caring about their own productivity and trying to improve it on an ongoing basis.

The irony is that I was using the same editor for decades. I did keep trying to find improvements, but never could. There are two sides to Emacs - one is as a competent & coherent editor, and the other is "living" in it. It has builtin web browsing, image viewing, email and news support, terminal emulators etc. I was never one of those.

Before Emacs I used vi. Its modal interface, small size, and availability on all systems make it a good tool. However it was text console only, and didn't do colour, menus, multiple files or other useful functionality. (It does now.) vi does have a learning curve - I estimate it takes about 4 years to be good with it, and 8 years to be an expert!

I had known about Emacs for a while, but it was text console only, and didn't do colour, or menus. Each attempt to use it left me frustrated with what amounts to another arbitrary set of keystrokes. (I've always been a cross platform person so I was also juggling keystrokes for other operating systems and applications.) A colleague (hi Jules) introduced several of us to XEmacs around 1995. It had a gui, and colour, and most importantly a menu system. It was no longer necessary to memorize a large set of new keystrokes, as the menus showed them. You could do everything without knowing any, and then pick up those you use often enough.

By the mid 2000s XEmacs was languishing, and Emacs was slowly catching up with the gui. More and more packages only worked with regular Emacs (there were small but growing incompatibilities). I eventually made the switch from XEmacs to regular Emacs.

There was an explosion in different file types I was editing: Python, C, Javascript, Java, Objective-C, HTML, HTML with Jinja Templates, JSON, matlab, CSS, build scripts, SQL, and many more I have forgotten. Emacs had support for most. Support means syntax highlighting, indenting, jumping around notable symbols etc. More packages were produced that did linting (looking for common errors), and various other useful productivity enhancements.

At the same time a new editor Sublime Text was introduced. It had fantastic new interaction (goto anything, projects, command palettes, multiple selections, distraction free) and a rich package system (written in Python - yay!) I kept trying it, but kept finding issues that affected me. Development also seemed to drastically slow, and since it was closed source there was no way for others to improve and update the core.

Meanwhile Emacs became more and more frustrating. The web (HTML, Javascript, CSS) is not a first class citizen. Not many packages were distributed with the core, and you had to copy crytic elisp code from various places, or use strange tools to try and get them installed and kept up to date. Then you had to do that on each machine. Heck the package repositories (eg MELPA) didn't even use SSL by default! My emacs configuration file kept getting longer and longer.

Ultimately tools these days are defined by their vibrant community, useful defaults, and easy to use extension mechanisms. Emacs has all those, especially in the past. But they are of a different era and different cadence.

I have switched to Atom. It had a rough initial exposure with performance problems, and the extremely dubious choice of being closed source. However both have been addressed. Just days before Atom 1.2 was released, I removed Emacs in favour of Atom 1.1. My configuration file is 10 lines long, and I get the same experience on every machine.

Category: misc – Tags: exit review


Developers should work in support

I am one of the many many people getting the completely useless response on trying to upgrade to Windows 10.

Windows 10 Error Message

Closing setup and trying again doesn't work. Nor did changing my language settings, .Net repair tools, freeing up huge amounts of space, rebooting, examining log files, or reading tea leaves.

A lesson I learned many years ago is that developers should work in support [1] (eg 3 weeks a year). Normally their experience of support issues is after they have been filtered through many layers of other people, and they don't see the ones that have been resolved even if fairly frequent. Essentially developers do not experience the friction that their customers or support staff regularly encounter.

I've done the support work myself, as well as seeing other developers doing it. There are immediate fixes such as tweaks to tools, asking for information in a different order, or new ideas for how to address common issues (hopefully eliminating them). Then they will go back and fix unhelpful messages in the product. For example a message like "couldn't find file" will be changed to say which file, and possibly detect if it is because the file doesn't exist or the directory containing it. Or make the code create a default file and parent directories.

From that point on, the developers produce substantially better diagnostics. They work out what information they would need on answering a support call and make the diagnostics provide it all. But as time passes, the memories fade, and shortcuts are taken. That is why working in support should be done regularly.

The Microsoft developers responsible for the screenshot "something happened" are likely a lost cause though.

As a side rant, Linux distributions are also distributed as ISO files. They can be used as is on optical media, network booted, dumped as is onto USB flash drives, and work on BIOS and EFI systems (even Apple's non-standard EFI). The Windows ISO is considerably more painful, especially if your machine doesn't have an optical drive, like pretty much all of them these days.

[1]This applies to larger companies. In small companies/startups you often end up with everyone doing support.

Category: misc – Tags: rants


Exploring two different battery wifi hubs

Fundamentals

I recently decided to get a multipurpose device. They can do all this:

  • Large battery to recharge other devices over USB (eg your phone and tablet)

  • Provide wifi access to a network in front of it

    A wifi network is provided behind the device with a name and password of your choosing. You connect one or more of your devices to that.

    In front of it you can have no network at all, a wifi network (unrelated to the one behind), or a wired ethernet network. You do have to configure access to the network, but only for this device. Your devices are behind it blissfully ignorant of the real network.

  • Exports attached storage (eg USB stick, USB hard drive, sdcard) via both SMB (aka "Windows network file sharing" supported by virtually everything these days) and DLNA (a multimedia network protocol, supported by many although the Apple ecosystem prefers "iTunes")

    On Android and desktop systems, you'll find that Kodi works for both SMB and DLNA, as does Android ES File Explorer (SMB only).

  • They are cheap ($40 - $60 depending on battery capacity)

  • Can run completely off the battery so no additional power is needed. They will run for many hours. They will also run while being charged.

  • Similar in size to a pack of cards

  • They use popular standards - eg they charge using standard micro-USB, provide power for devices with standard USB port, use existing filesystems, standard protocols etc. There is no need to carry different cables or chargers, and any software speaking SMB or DLNA works.

After some agonizing over Amazon reviews, and reading the manuals, I ended up with two.

Photo of both products

The left red one is a HT-TM05 TripMate Versatile Wireless N Travel Router (Amazon page) although the packaging and internal names say Tripmate Sith. The right white one is a RAVPower RP-WD02 Wireless Filehub / Portable Travel Router (Amazon page). They are sold by the same company, and the underlying products are substantially similar except for the hardware layout.

How they do all this turns out to be quite simple. The battery provides power, and there is a small Linux based computer attached. It is running a MIPS based processor (the manuals even tell you the exact manufacturer and model number), 32MB of RAM, and 8MB of builtin storage for their software. For some reason MIPS cores seem very popular in network access devices - if you have a box at home from the likes of Linksys, DLink, Netgear etc, it is almost certainly using MIPS.

Praise

They fundamentally do what they say. Both RAVPower and Hootoo provide Android and iOS apps to help access and configure the devices. However neither requires it and you can do all the configuration work in a web browser by going to the device address (default 10.10.10.254). It looks like the apps are really just some logic to find the device on the network, and then show the admin pages in a WebView. Note that I have never tried the apps.

Each device has some nice highlights the other doesn't. (If only someone made something combining the best of both.) The Hootoo has some lights on top to see battery level (they only light when you press the button as I did before taking the photo). The RAVPower has a micro-sdcard slot. The Hootoo can stand up. The RAVPower has a label giving default username, passwords and IP address. The Hootoo web admin pages are nicer, simpler and mobile optimised. The RAVPower ones tell me the device's external IP address. The Hootoo's lights go on or off in sequence during power on and power off so you have progress feedback.

As a test I left the HT-TM05 10,400mAh device on and connected to the wifi network. I didn't have anything connected to it, so this is a measure of the longest it can continuously run. After 45 hours (3 hours short of two full days) it had dropped to one battery led (out of four), and I decided to recharge it rather than deplete the battery completely. That is an impressive runtime. The RTP-WD02 has a 6,000mAh battery so you would expect a proportionate maximum runtime around 28 hours.

Suggested Improvements

The RAVPower has ports on 3 sides, which can lead to cables sticking out in all directions. The Hootoo is nicer with ports on two sides next to each other. Sadly the micro-USB for charging is right next to the USB for connecting storage. If the cables connecting either are anything but skinny heads then you can't have both connected. If you use an sdcard reader on the Hootoo then it will overlap the charging port. You get a choice of too dense ports (Hootoo) or not dense enough (RAVPower).

Hootoo really should have a builtin sdcard reader.

The web admin UIs have no help. When you want to safely remove attached storage, you'll end up at a page with a button labeled "Delete". It takes a lot of courage to press the button, to confirm that it really means "remove" or "eject" (it does). Firmware updates on both devices added an "auto jump service", you can enable or disable. Good luck on figuring out what that does!

Censure

Software versions

It didn't take me long to get access into the devices. Here is what the Hootoo said it is running:

$ cat /proc/version
Linux version 2.6.36 (gcc version 3.4.2) #8 Fri Jul 11 10:44:45 CST 2014
$ /usr/sbin/smbd --version
Version 3.0.24

RAVPower:

$ cat /proc/version
Linux version 2.6.21 (gcc version 3.4.2) #5 Fri Nov 1 13:36:46 CST 2013
$ /usr/sbin/smbd --version
Version 3.0.24

The Linux kernels date from 2007 and 2010. Neither version is long term supported, and both have various known security holes, although remote security holes are very rare.

smbd is the main component of Samba and provides networked file access. Version 3.0.24 was released in 2007, and there have been numerous releases since then, including 3.0.25 a few months later which fixed 3 security holes. Virtually all Samba security holes are remote since that is what it does.

I didn't check the versions of other accessible services (eg DLNA server, NTP), but this pattern of older versions with known problems is most likely. (The gcc version above is from 2008.)

Network exposed

Why do the versions matter? Both vendors (RAVPower update) made a very bad decision - all network services including the web admin pages, Samba, DLNA, and even a telnet server are accessible from in front of the device. If for example you are at an airport, campus, coffee shop, hotel or somewhere else with a network, and connect the device, then anyone on those networks can connect to the network services on the device. They do not need to connect to the wifi on it. A bad guy has more than 5 years of published security holes to choose from, and can have complete control over it. (The default usernames and passwords also make this a breeze.)

Complete control means they can extract your saved wifi password (eg if last on your home network, or for the current network), redirect or monitor your traffic, replace the firmware etc. To a certain extent this is no different than connecting to someone else's network which you have to assume is hostile, but this is something that goes around with you. (Both vendors use the word 'secure' in their Amazon descriptions.) While that kind of exploitation sounds far fetched, bad guys are already doing it.

Bridge mode

Both products' Amazon pages claim to support a bridge mode, but this marketing fluff and not the term as understood by networking people. They never bridge in the sense that those behind the device and the network in front are joined making a unified LAN. The devices always do network address translation (NAT) and never any form of bridging.

Admin Pages

As far as I can tell, Hootoo are the firmware developers. Their older products as well as the RAVPower use a fairly clunky web interface. It looks like a singe page application but doesn't do it well.

The Hootoo has a newer web interface where the URL changes as you navigate around pages, making it much easier to see what is going on, send links to others or other devices etc. It is also mobile centric giving the same pages that look good on a phone, as to a large monitor.

I had a quick look at authentication to see if there were any simple holes. Both use their own login screen, which means your browser can't prompt you nor remember the password. They set a session id cookie and require it to be present for other web accesses.

The pages are always over http, and not https, although there isn't much of an alternative. (Browsers are getting very hostile to self signed certificates.)

Both devices ended up with a second web server on port 81 (standard http is port 80), that appears to be related to the admin server. There is no need for it, and I'd be concerned about what it does.

Many changes cause the device to reboot and your browser to show a many minute "please wait" message. This gets very annoying. I understand why it is done (far simpler to code and test), but not doing it so much would be a more pleasant experience.

Firmware updates require storage to be connected as the devices don't have temporary storage. On both devices they also wiped out all settings.

RAVPower update

20 May, 2015

I sent an email to RAVPower support around the network exposing and GPL issues. There was no response. A few days later there was a comment on my Amazon reviewing asking me to email support, so I did a second time.

They claimed the issue had been fixed with new firmware, and a pointer to some source. I can confirm that the new firmware does indeed stop exposing network services to the public.

The source link was to Hootoo's website and looked like an effort had been made for some GPL awareness. It included a document outlining components, their version numbers, and license. It also included the kernel source code and Samba (including patches). I did verify the kernel and Samba versions matched, but did not verify they could be built or were exactly what was on the device (both GPL requirements). There didn't appear to be much other source present.

I did have more interaction with support, who didn't understand the difference between telling me about that source drop and actually complying with the GPL. It needs to be available to all users (without having to ask), requires copyright notices be present, be complete and more.

Hootoo update

28 May, 2015

Email to Hootoo support went unanswered. However I did see new firmware appear, which claimed to add exFAT support.

On the network exposed front, the telnet server was disabled, but another web admin server appeared on port 81.

Category: misc – Tags: review


Prisoners are people

The US penal system is despicable in many different ways. John Oliver covers some of it.

Category: misc – Tags: reality

Contact me