Wednesday, March 27, 2013

a Perl module on CPAN for dealing with ZIP archives: IO::Compress::Zip

IO::Compress::Zip - Write zip files/buffers - metacpan.org

I found code, that makes use of this module. The 1st problem, I had to deal with: the file to be written into the ZIP archive and the ZIP archive both get named using absolute pathnames. That's alright for the ZIP archive, but you usually don't like archive members with absolute pathnames. The manual page describes, which feature deals with this ("FilterName"), and it also gives a nice and short recipe.

the "Table editor for Emacs"

Table

My minor ongoing problem with it: Whenever I copy an area from a table cell, the buffer gets flagged as "modified". That's ugly, because afterwards you wonder, what got changed and you don't see it of course.

The demise of Google Reader: Stability as a service

The demise of Google Reader: Stability as a service:
Om Malik’s brief post on the demise of Google Reader raises a good point: If we can’t trust Google to keep successful applications around, why should we bother trying to use their new applications, such as Google Keep?
Given the timing, the name is ironic. I’d definitely like an application similar to Evernote, but with search that actually worked well; I trust Google on search. But why should I use Keep if the chances are that Google is going to drop it a year or two from now?
Google Keep screenshotGoogle Keep screenshot
In the larger scheme of things, Keep is small potatoes. Google is injuring themselves in ways that are potentially much more serious than the success or failure of one app. Google is working on the most ambitious re-envisioning of computing since the beginning of the PC era: moving absolutely everything to the cloud. Minimal local storage; local disk drives, whether solid state or rust-based, are the problem, not the solution. Projects like Google Fiber show that they’re interested in seeing that people have enough bandwidth so that they can get at their cloud storage fast enough so that they don’t notice that it isn’t local.
It’s a breath-taking vision, on many levels: I should be able to have access to all of my work, regardless of the device I’m using or where it’s located. A mobile phone shouldn’t be any different from a desktop. I may not want to write software on a mobile phone (I can’t imagine coding on those tiny touch keyboards), but I should be able to if I want to. And I should definitely be able to take a laptop into the hills and work transparently over a 4G network.
Furthermore, why should I worry about local storage? The most common cause for throwing a computer on the bone pile is disk drive failure. Granted, I keep machines around for a long time, so by the time the disk drive fails, it’s more than time for an upgrade. But local disks require backups; backups are a pain; and it’s all too common for something to go wrong when you’re doing a restore. I’d prefer to leave backups to a professional in a data center. For that matter, there are many things I’d rather leave to a data center ops group: malware detection, authentication, software updates, you name it. Most of the things that make computing a pain disappear when you move them to the cloud.
So I’ve written two paragraphs about what’s wonderful about Google’s vision. Here’s what sucks. How can I contemplate moving everything to the cloud, especially Google’s cloud, if services are going to flicker in and out of existence at the whim of Google’s management? That’s a non-starter. Google has scrapped services in the past, and though I’ve been sympathetic with the people who complained about the cancellation, they’ve been services that haven’t reached critical mass. You can’t say that about Google Reader. And if they’re willing to scrap Google Reader, why not Google Docs? I bet more people use Reader than Docs. What if they kill the Prediction API, and you rely on that? There are alternatives to Reader, there may be alternatives to Docs (though most of the ones I knew have died on the vine), but I don’t know of anything remotely like the Prediction API. I could go on with “what ifs” forever (Authentication API? Web Optimizer?), but you get the point.
If Google is serious about providing a platform that lets us move all of our computing to the cloud, they need to provide a stable platform. So far, the tools are great, but Google gets a #fail for stability. Google understands the Internet far better than its competitors, but they’re demonstrating that they don’t understand their users’ needs.

my Android deviced used for tethering – where is the respective log file?

It feels like the device is experiencing quite a few (unexpected) disconnects on the 3G ("WAN") port. I suppose there is a log file, where I can find the respective event entries. Where would I find this log file? Any idea anybody?

On my new LTE router (an AVM FRITZ!Box 6842 (maybe this link still leads you to its elder sister 6840) [German link])

de-obfuscated again – "Berliner Sparkasse" web banking access reverse-engineered

A couple of weeks after "Berliner Sparkasse" renovated their online banking portal, I managed to amend my automated account statement download to work with their portal again. Actually it is not just amending, I rather had to write it again. Alright, I actually gained some experience in that area through the years, but it still is rather a hard and frustrating business.

Their portal uses obfuscated HTML, so it's not a trivial job, and you never really know in advance, whether you invest your time properly and successfully in that reverse engineering job. (At some stage I sometimes think, I will give up.) To put things right: this is not a job I am getting paid for, it just eases my task of regular and frequent downloading the statements of all bank accounts belonging to "family and company".

I would be terribly honoured to get invited to present my page scraping and de-obfuscation approach at some workshop or conference. If  the circumstances of the invitation would allow me to not loose money by missing on my bread-and-butter job, that would be so welcome!! If de-obfuscation and page scraping would even become my special niche to make a little money on, maybe fund my sons' education and maybe even avoid the hardnesses of expected old-age poverty – oh, paradise!!!

No, actually I don't really want to help the "obfuscation community" to improve their obfuscation frameworks, but then … – you should understand: on my side this is a one-man-show, they get backed by the financial industry and whoever.

Thursday, March 21, 2013

VirtualBox 4.2.10 fixes a number of crashing bugs

VirtualBox 4.2.10 fixes a number of crashing bugs: The latest version of Oracle's open source virtualisation package fixes several crashing problems and other bugs, including compilation problems with the rc0 release of version 3.9 of the Linux kernel


Wednesday, March 20, 2013

Safari: .NET Bibliography

.NET Bibliography:
Take advantage of this bibliography to learn everything about .NET programming, from C# to VB.NET and ASP.NET to WCF and WPF to XAML and Kinect.

I assume, you get this book for free.

munpack = MIME unpack – a utility to unpack messages in MIME or split-uuencode format

munpack(1) - Linux man page

If you need to deal with MIME attachements of an e-mail message within a shell script, this has been the tool for this purpose for ages now.

O'Reilly Media book: Mac Hacks

Mac Hacks:
OS X Mountain Lion is an incredibly powerful, but if you’re a serious Mac user who really wants to take control of this operating system, this book helps you dig below the surface. Tweak system preferences, mount drives and devices, and generally do things with your system that Apple doesn’t expect you to do.

http-console — a REPL loop for HTTP

cloudhead/http-console · GitHub

REPL = "read–eval–print loop":
A read–eval–print loop (REPL) is a simple, interactive computer programming environment. The term is most usually used to refer to a Lisp interactive environment, but can be applied to command line shells and similar environments for programming languages such as …

Working With WebSockets (by Robert Hall on Safari Books Online: Publishing & Technology)

Working With WebSockets

O'Reilly Media book: OAuth 2.0: The Definitive Guide

OAuth 2.0: The Definitive Guide:
There are currently no other books focusing on the OAuth 2 protocol. OAuth is mentioned in some other books about APIs and building social applications. The only other resources on the topic are the spec itself, API documentation from some service providers, and a few blog posts by core authors of the spec.

Xming is an implementation of the X Window System for Microsoft Windows operating systems

Xming - Wikipedia, the free encyclopedia

The Xming X server is based on Cygwin/X, the X.Org Server.
Cygwin/X  is the X server, I had used occasionally on Windows.

I heard of Xming last night at the etengo Hamburg freelancer meet up.

Wednesday, March 13, 2013

"unit testing" terminology


  • on the outer level: startup + shutdown
  • on the inner level: setup + teardown

Emacs: tabs vs. spaces – how to make all indentation from spaces only

Just Spaces - GNU Emacs Manual

(setq indent-tabs-mode nil)

resp.

M-x set-variable …

resp.

# Local variables:
# indent-tabs-mode:nil
# End:

Saturday, March 9, 2013

my 1st concurrent backups from my Mac and my openSUSE Linux VM to the Synology DS213+

With the "alternative rsync" (see my older article on rsync and Synology!) things started looking a little more successful.

Apparently a lot of files by the name "@eaDir" get created "automagically" in various subdirectories. And as I usually start rsync with "--delete-after", they all get removed in the end. I am not sure, whether the DiskStation OS ("DSM") likes that, so for the time being I will do w/o that command line option. I will investigate the "@eaDir" issue "in the near future" (TBD).

Another issue: lots of messages like "cannot chgrp …". Apparently that's because the group identifier on the source system does not exist on the target system. I definitely don't want to see that message, so something must be done in that area (TBD).

It really is a relief to have a regular backup procedure in place again.

"rsync over ssh" to a Synology NAS – IPKG, opkg

Nota Bene / before:
  • This is the text, that helps me personally re-installing rsync, whenever it magically disappears, maybe caused by an DSM update.
  • True, I should bring a little structure into this article.
  • I never ever care about rsyncd, as I am doing "rsync over ssh";  any activity suggested regarding rsyncd does not concern me. (But then sometimes I am misled, and I erroneously do invest time in false paths, and I do get into problems that are actually rsyncd-related.)
  • I have my own ways of backing up data.
  • I have my own ideas of keeping data long term outside my computer's built-in disks.

The Synology system provided rsync is not for me. It's somehow modified and spoilt.

backup - Rsync over ssh: "ERROR: module is read only" suddenly appeared - Server Fault

Well, it's not only me experiencing weird effects with the "built-in" rsync. And I think, quite a few of the answers to that forum article do not quite fit at all.

The details of the case:
  • "rsync over ssh" does not use the remote rsync daemon at all, i.e. no rsyncd.conf applies
  • it's not unlikely, that these rsync problems have to do with a Synology special version of rsync
  • after this: "DiskStation> /usr/syno/etc.defaults/rc.d/S84rsyncd.sh stop", the error message does not change, so it's really not an "rsync daemon" issue, as that one is no longer running
I fixed this problem by installing an rsync package provided on an Unslung Project repository for the CPU used within my Synology NAS. This brings dependencies on the Unslung Project, their IPKG installer, their Optware, their specific Synology repository. This kills me a little, to be honest.

I mainly followed the "German Synology wiki" on IPKG. (Anxiously as I am, I do keep a backup of that article at …/com/synology-wiki.de/ . Yes, I do have an idea, that this step alone is rather in vain.)
So, well 1st I had to install IPKG itself.
My CPU on the DS213+ is an e500 (actually: e500v2), so I went the e500 way in their description.
My CPU on the DS112+ is a "Feroceon 88FR131 rev 1 (v5l)", there is a note on how to deal with the DS112+, pls follow that one.

Then I installed the rsync package provided by IPKG.

An entry for /etc/rc.optware got created in /etc/rc.local, and they suggest to remove it, and I did that.
I am not sure, it concerns me at all, as I mention "my" rsync on the NAS by full path on my command lines outside the NAS.

Now my only minor problem with the "alternative rsync" is, that I have to specify the remote rsync on my local rsync command lines like this: "--rsync-path=/opt/bin/rsync". That's a PITA, but that's not too hard.

Update 2013-08-31: en.wikipedia.org/wiki/Ipkg says IPKG was discontinued. en.wikipedia.org/wiki/Opkg is the follower, code.google.com/p/opkg/, but it hasn't really taken off.

the wikis for the Synology DiskStation

Friday, March 8, 2013

AIX has a limit for cronjobs concurrently being run

Sounds reasonable in a way. But it's rather bad, if you are not aware of it, and your corporate OS Monitoring Services do not remind you of critical entries in the resp. cron log file. They may say something like:
there is crontab entry, that did not get executed, because we exceeded the maximum number of … – maybe you want to fix something here
I suppose, similar limits exist in every UNIX flavour.

upgrading the firmware on my new NAS (Synology DS213+) to DSM 4.2


Thursday, March 7, 2013

O'Reilly Media book: Take Control of Your Passwords

Take Control of Your Passwords:
Improve your passwords without losing your cool, thanks to Joe Kissell's expert advice. Start on the path to modern password security by watching Joe's intro video and by checking out our "Joe of Tech" comic in the Contents & Intro tab below (scroll down!).
Read the book to understand the problems and apply a real-world strategy that includes choosing a password manager, auditing your existing passwords, and dealing with situations where automated tools can't help.
"Awesome. You did an amazing job breaking it down. This should be mandatory reading." --Rich Mogull, CEO at Securosis
This ebook helps you overcome frustrations that arise when attempting to design a strategy for dealing with the following password problems:
  • 9-character passwords with upper- and lowercase letters, digits, and punctuation are NOT strong enough.
  • You CANNOT turn a so-so password into a great one by tacking a punctuation character and number on the end.
  • It is NOT safe to use the same password everywhere, even if it's a great password.
  • A password is NOT immune to automated cracking because there's a delay between login attempts.
  • Even if you're an ordinary person without valuable data, your account may STILL be hacked, causing you problems.
  • You can NOT manually devise "random" passwords that will defeat potential attackers.
  • Just because a password doesn't appear in a dictionary, that does NOT necessarily mean that it's adequate.
  • It is NOT a smart idea to change your passwords every month.
  • Truthfully answering security questions like "What is your mother's maiden name?" does NOT keep your data more secure.
  • Adding a character to a 10-character password does NOT make it 10 percent stronger.
  • Easy-to-remember passwords like "correct horse battery staple" will NOT solve all your password problems.
  • All password managers are NOT pretty much the same.
  • Your passwords will NOT be safest if you never write them down and keep them only in your head.
"Joe handles a confusing and scary subject more clearly and calmly than I would have thought possible. I'll be recommending this book to just about everybody I know." --William Porter, database developer, author, photographer

Wednesday, March 6, 2013

O'Reilly Media book: ZeroMQ – Messaging for Many Applications

ZeroMQ:
Discover why ZeroMQ is rapidly becoming the programming framework of choice for exchanging messages between systems. With this practical, fast-paced guide, you’ll learn how to use this lightweight and highly flexible networking tool for message passing in clusters, the cloud, and other multi-system environments. Created by ZeroMQ maintainer Pieter Hintjens and volunteers from the framework’s community, this book takes you on a tour of different real-world applications, with extended examples in C.

O'Reilly Media book: Head First Python

Head First Python:
Add Python to your programming skills and have some fun at the same time. Head First Python takes you beyond typical how-to manuals with an engaging visual format that includes images, puzzles, stories, and quizzes that are proven to stimulate learning and retention. You’ll not only learn how Python differs from other programming languages and how it’s similar, you’ll learn how to be a great programmer.

O'Reilly Media book: Opa: Up and Running – Rapid and Secure Web Development

Opa: Up and Running:
Want to simplify web development? This hands-on book shows you how to write frontend and backend code simultaneously, using the Opa framework. Opa provides a complete stack for web application development, including a web server, database engine, distribution libraries, and a programming language that compiles to JavaScript.

O'Reilly Media book: Packet Guide to Voice over IP

Packet Guide to Voice over IP:
Go under the hood of an operating Voice over IP network, and build your knowledge of the protocols and architectures used by this Internet telephony technology. With this concise guide, you’ll learn about services involved in VoIP and get a first-hand view of network data packets from the time the phones boot through calls and subsequent connection teardown.

O'Reilly Media book: Learning Java

Learning Java:
If you’re new to Java, the fourth edition of this bestselling guide provides an example-driven introduction to the latest language features and APIs in Java 6 and 7. Advanced Java developers will be able to take a deep dive into areas such as concurrency and JVM enhancements.

Gregor Purdy's "ltag" perl script lists the symbolic tags … a CVS sandbox directory

Gregor Purdy - Software

Gregor is the author of the o'Reilly book "CVS Pocket Reference" [o'Reilly link].

Saturday, March 2, 2013

my new NAS (and so much more): Synology DS213+

DS213+ Products - Synology Inc. Network Attached Storage

The box makes itself known as diskstation.local resp. DiskStation on my LAN. (Reliability …)

Of course it deserves a "1 Gbit/s" on my FRITZ!Box.

The box's OS is BusyBox; here are a few more details:


DiskStation> uname -a
Linux DiskStation 2.6.32.12 #2668 SMP Tue Dec 11 13:01:16 CST 2012 ppc GNU/Linux synology_qoriq_213+


I enabled SSH access for my personal user account rather soon, but the SSH login was not instantly successful. I tried "ssh -v", and from there the login seriously looked, as if it was almost successful. My web research wasn't successful either. Rather desperately I enabled "telnet access"; I tried to log in, and this was the result:
login: can't run /sbin/nologin: No such file or directory
It was clear then, that the default shell in the /etc/passwd entry for my user account was in my way /sbin/nologin so far. I made /bin/ash the default shell and everything was fine. Of course, I disabled telnet access again thereafter. And of course, once I added "nologin" to my web query, I found a few articles describing the issue.

SCP works; "rsync over ssh" (with my personal user account) does not work yet:

$ rsync -va --rsh='ssh -l root' /etc/motd diskstation:tmp/ # works$ rsync -va /etc/motd diskstation:tmp/ # fails
sending incremental file list
ERROR: module is read only
rsync error: syntax or usage error (code 1) at main.c(1034) [Receiver=3.0.8]
rsync: connection unexpectedly closed (9 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(605) [sender=3.0.8]
Fiddling with /etc/ssh/sshd_config – I really don't know, what to change in order to overcome this obstacle.


Perl is not part of its standard equipment, but Python apparently is,

Through their Download Center I found this FTP server link. You can find various software for their boxes on their. There are also Perl installation packages, but the ones I tried are not for the current firmware release of "my" box, and in a way I am not up for an upgrade to beta software for my NAS.

(For now I summarised here everything I have tried so far, so I will be able to procede from here.)

A few more links:
There are suggestions on the wiki(s) to install ipkg (yet another packaging system, not officially supported by Synology), and to install another rsync through ipkg. I may consider this, but I am not yet sure. I am quite able to use rsync (over ssh) as user root, but I am not with my personal account. I have no clue, what rights it needs, to allow my personal account to work as smoothly as user root. (To Be Solved!)

To be continued …

AVM's LTE Router: FRITZ!Box 6842 LTE

FRITZ!Box 6842 LTE

As opposed to its elder FRITZ!Box LTE siblings (6810, 6840) this one supports 3 bands:
  • 800
  • 1800 – that's the band, that Deutsche Telekom supports in German major towns like Berlin and Hamburg
  • 2600
conrad.de announced the availability on … for …:
  • 2013-03-22: "available"
  • … 2013-03-11: 2013-03-13
  • 2013-03-13 .. ___ : 2013-04-03
Update 2013-03-27:
Good news: You don't need a dedicated LTE SIM card provided by you telecom company, I plugged the SIM card from my 3G Android tablet into this FRITZ!Box, and it just worked.
I love to see the familiar web GUI of the AVM FRITZ!Box.
Rather interesting and actually annoying to see the many LTE disconnections logged by the FRITZ!Box. Maybe the technical connection quality going through the LTE FRITZ!Box is not really better than going through the 3G Android tablet. Maybe I should have sought the 3G connection / disconnection / tethering log file on the tablet a little better. But seeing this log displayed on the FRITZ!Box GUI is just overwhelming. Maybe one day I will use it not just as a 4G Mobile Internet router.

Friday, March 1, 2013

my new file server

Producers
  • synology.com (preferred)
  • qnap
  • drobo (good, but too expensive)
They all offer some kind of flexible disk mirroring with non-identical disks resp. disk sizes.

Main criteria:
  • mini tower, not rack

My colleague's good advice: Use Samsung disks!

Apparently "they all" have hardware compatibility lists, i.e. they don't work with every disk.

I think, it will be this one: Synology Diskstation DS213+ NAS System 6TB (2x 3TB WD Red für NAS WD30EFRX). This is the one I got today.