Code by Kevin

About
Code by Kevin, Programming, code, business, and other pursuits

Your Host
Kevin Walzer, software developer.



Home

Subscribe to RSS Feed
Get a syndicated feed of my weblog.


Archives
2014
2013
2012
2011
2010
2009
2008
2007
2006

Categories
Business
Software
General

December 2014
Sun Mon Tue Wed Thu Fri Sat
 
     

Privacy Policy

Site design: Skeleton

 

Tue, 16 Dec 2014

TkWebView 1.1

I've released version 1.1 of TkWebView, my Tcl package that allows a developer to add a native Cocoa WebView/WebKit widget to a Tcl/Tk application, using the same rendering engine that powers Safari.

It is designed for display of local HTML files. Links to remote sites (i.e., those beginning with http://) will launch the user's default browser. TkWebView does not display remote URL's; this is not a technical constraint but rather a design decision, to encourage a separation between local display of HTML content and an external browser. TkWebView also does not provide direct access to the HTML content, browsing history, etc. from Tk; it delegates all management of browsing history and HTML content to WebKit. HTML navigation can be built into the HTML pages themselves via JavaScript, href links, and so on.

This release of TkWebView adds improved support for running JavaScript within the WebView, such as buttons that assist with navigation; it also adds support for printing the contents of the WebView to a file or to a printer.

TkWebView is available under a BSD-style license.

[/software] permanent link

Thu, 04 Dec 2014

ruby2app, a tool for bundling Ruby scripts into standalone Mac applications

I'm pleased to announce the initial release of ruby2app, a tool for bundling Ruby scripts into standalone Mac applications:

https://github.com/codebykevin/ruby2app

ruby2app is a simple command-line tool, inspired by similar Mac tools like py2app for Python, that bundles a Ruby script with the Ruby runtime into a standalone Mac application. It reads a basic configuration file to find the appropriate Ruby installation, compiles a small C program that embeds the Ruby interpreter, copies the entire Ruby installation into the application bundle, and re-links the relevant libraries with the main executable. Ruby must be compiled with the "--enable-load-relative" flag for ruby2app to work.

At present the tool is primarily geared for Ruby-Tk applications, since Tk is the best-supported open-source GUI framework with Ruby bindings on the Mac in 2014, and (not coincidentally) is the one I use. But I invite others to submit patches to add support for other UI frameworks as well; Qt support might be a good candidate. Support for RubyMotion is outside the scope of ruby2app, as RubyMotion presumably has its own toolchain.

I am still in the process of learning Ruby, and in fact wrote this tool to be certain that I could deploy a Ruby-Tk application in a convenient manner on the Mac desktop. There are no other current tools that provide basic support for deployment of an open-source Mac application written in Ruby. Ruby-Cocoa and MacRuby are effectively dead and their tools (standaloneify.rb) are obsolete, RubyMotion has its own toolchain, and crate and rubyscript2exe do not support current versions of Ruby.

I do have a decade of experience writing Mac applications in Python, Perl and Tcl using the Tk toolkit, so I adapted some ideas from bundling tools for those languages in creating ruby2app. ruby2app is written in Bash, and thus should be easy to read and modify as required, and should fit in well to any application building toolchain.

ruby2app is available under the MIT license, and feedback is welcome.

[/software] permanent link

Wed, 26 Nov 2014

FileMorph 2 now on App Store

FileMorph 2.0, which I announced a couple of weeks ago, is now available in the Mac App Store as well as from my own website.

This is a significant announcement because it means that all private API's have been removed from Tcl/Tk, as verified by Apple's own scanning tools, and therefore apps using Tcl/Tk can be submitted to the MAS using up-to-date versions of Tcl/Tk and without linking to Apple's own outdated system installation. As I noted previously, I spent several weeks scrubbing the private API calls, forbidden by Apple, from Tcl/Tk. One reason for the lag between the release of FileMorph 2.0 and its acceptance in the MAS was that there was still a bit of private cruft that I had overlooked, which has since been addressed.

Going forward, I plan to keep my apps as up-to-date as possible with Tcl/Tk, and not link to Apple's libraries any more.

[/software] permanent link

Wed, 12 Nov 2014

FileMorph 2.0

I've release FileMorph 2.0, an app I created in Tcl/Tk a few years ago to provide batch modification of file names and attributes, such the last date accessed. My intent in creating the program was to offer an easier-to-use tool than other file renaming programs, which tend to overwhelm with their options, and the command-line, which is powerful but obscure. I withdrew FileMorph from sales a year or so ago to focus on some other projects, but decided to return to it this fall as part of my effort to add Perl as a development language to my toolkit: FileMorph is an ideal app to implement in Perl because its functions play right into Perl's strengths, with is text and file management.

I'm pleased with the results overall. Perl has proven to be a pleasant language to develop in, and its bindings to the Tk toolkit (via its Tkx module) and the availability of the Mac::QuickBundle module make the development of a full-featured, GUI-based, desktop app feasible.

There were a few hiccups in the development process that led to it taking longer than expected. First, there was the usual assortment of changes in Apple's development tools and requirements for submitting to the Mac App Store, which caused some delay. Also, there were a few unexpected snags in Perl's interaction with Tcl under the hood: occasionally, when running a Tcl/Tk function from Perl, I would get obscure errors like "sh: -c: line 0: unexpected EOF while looking for matching `''". In Perl code, this typically means that there is a typographical error, a missing quote mark that causes execution of code to stop; but when the error occurs in the context of a Tcl procedure that Perl is calling, the actual issue is anyone's guess. I observed this when Perl (via Tcl) was downloading a file from the Internet, and parsing XML (part of the application update machinery I've implemented). The only solution was to continually tweak the code until the error no longer occurred, even though it wasn't clear to me where the error was.

I'm using this version of FileMorph as the basis for the next generation of my development platform, as well, which means, among other things, that I am now moving from linking to the Mac-installed system Tcl/Tk to shipping my own, more up-to-date versions, stripped of the private Cocoa calls that previously caused rejection by the Mac App Store. This app also includes some UI refinements that better fit in with OS X 10.10.

FileMorph is now ready for sale at my website and is undergoing review at the Mac App Store. If you do batch renaming of files, it's worth a look.

[/software] permanent link

Mon, 13 Oct 2014

Ruby and Tk 8.6

I've written about my efforts to add Ruby to my desktop development toolkit, particularly to make it feasible to deploy standalone Ruby-Tk apps on the Mac. My efforts with Ruby deployment had paid off with a viable approach, and I was feeling confident about using Ruby as a language for a planned project.

Imagine my frustration, then, when I discovered that Ruby did not support the latest version of Tk, 8.6; numerous developers and users were reporting crashes. While this may be expected when a language like Tcl/Tk does a major new release, Tk 8.6 has been out for nearly two years, since December 2012. It would seem that two years is more than enough time to work through whatever bugs and incompatibilities might arise, but apparently the issues were rather complex.

I haven't yet migrated my Tcl/Tk apps to 8.6, mainly because Tk's use of private frameworks on the Mac prevented me from using a version other than the one Apple ships with the operating system. Now that I have removed that code from Tk, migrating my apps to Tk 8.6 is going to be part of the next major releases of my products. Tcl/Tk 8.5 is likely to reach end-of-life status soon, meaning that it will no longer be supported or updated by the Tcl/Tk developers. As a result, if I want to include Ruby in my projects, it is vital that Ruby support 8.6.

Happily, after I posted a query on the Ruby-Talk mailing list about Tk 8.6, Ruby-Tk's maintainer, Hidetoshi Nagai, committed a patch to Ruby's development branch that adds "experimental" support for Tk 8.6. I rebuilt Ruby against Tk 8.6 and tested it against a complex Ruby-Tk application, and it seemed to run just fine--no crashes. That is terrific news, because it provides a foundation for full support for Tk 8.6 in Ruby, and it means I can add Ruby--a beautiful language with a rich library--to my app toolbox. I am very pleased, and encourage others to give Ruby-Tk 8.6 a try when the next version of Ruby is released. Thanks to Hidetoshi Nagai for being an extremely responsive developer.

[/software] permanent link

Wed, 24 Sep 2014

Private API's and open-source projects

I've been doing a great deal of wrestling lately with a paradox: the impact of including private Apple Application Programming Interfaces (API's) in an open-source project.

For the past few years I've been the chief maintainer of the Tk GUI toolkit on the Mac. I've been a Tcl/Tk developer for a decade, started coding my own platform-native Tcl/Tk extensions in C and Objective-C about five years ago, and took over maintainer duties in 2011 after the author of the Cocoa port of Tk, Daniel Steffen, was hired by Apple. Daniel had been the lead maintainer of Tk on OS X for several years before I assumed the role.

One of the key parts of the Cocoa port of Tk was its use of private, undocumented Cocoa API's for rendering of application windows and widgets; Daniel added this into Tk to help make sure that Tk rendered graphics quickly and smoothly. He borrowed the technique, and likely a lot of code, from WebKit, the Apple-sponsored, open-source HTML framework that powers the Safari, Chrome and other web browsers. This code provides the WebKit developers, and Tk developers, a great deal of precision and low-level control over the drawing of GUI elements, more control than is typically afforded by the Cocoa frameworks. I can't speak for how WebKit works, but Tk's overall architecture requires a lot of low-level control over the layout of a GUI. On Windows, and on the older Mac Carbon frameworks, Tk's design lines up nicely with the design philosophy of those toolkits; it's a very different situation with Cocoa.

There's a pretty significant problem with including private API's: Apple strongly discourages it, with good reason. Private API's are generally considered by Apple to be internal to the operating system, and are subject to change and even removal. It's an inherently unstable situation, and Apple goes beyond simply discouraging the use of such code; any application calling into private API's or frameworks is rejected from the Mac App Store (and also the iOS store for mobile apps). This means that Apple will reject apps that bundle code from WebKit; the same is true for apps that bundle code from Tk-Cocoa. The only workaround is to link to the (often older) frameworks that are shipped with OS X; private API's are OK there because they are installed by Apple.

I've been increasingly uncomfortable with Tk's use of private API's, but it wasn't until this year that I felt knowledgable enough to try to remove them. Eventually, that's what I did. I was tired of not being able to ship up-to-date Tk code in my own applications in the Mac App Store, and more importantly, I felt that it was a bad idea to have an open-source library such as Tk resting on such a fragile foundation.

And now, having stripped out the private API calls, I can see why they were included.

I must be candid: removing the private API's did result in a degradation of Tk's drawing performance under Cocoa. Rendering that was snappy and accurate became glitchy, slow, and laggy, with weird artifacts like buttons scrolling outside their container window and scrollbars appearing in two different places when a window was remapped. Sometimes widgets would not remap after a window resized, and in a few instances too much resizing would cause a crash.

A lot of the work I've done over the past couple of months, in consultation with a couple of contributors, has been to try and mitigate the worst effects of removing the private API calls. The button and scrollbar issues have been fixed, and some improvement in rendering with window resizing and resizing of child windows is now in place. For instance, I've added some code to skip over some drawing operations in window resizing, which has smoothed things out a bit. Basic user interfaces, such as the ones in the Tk demo, render very well with little discernible loss of performance. Stability seems fine.

Unfortunately, there remains some laggy drawing in more complex interfaces. I've spent a lot of hours over the past several weeks becoming acquainted with Tk-Cocoa's drawing code, and I regret that I can't wring out much more improvement here. Those private API's are undocumented and I don't fully understand how they worked or what they actually did, but they did add a lot of low-level magic to drawing Tk widgets. If getting the best performance for graphic rendering is the goal, including those bits is absolutely the right call, and I fully understand why Daniel Steffen included them. If an Apple-sponsored open-source project (WebKit) includes this code, then it's entirely reasonable for another Apple-sponsored open-source project (Tk-Cocoa) to follow suit.

From a policy standpoint, however, such design is untenable, because Apple is so strictly enforcing against the deployment of code that makes use of such design.

It's hard for me to get past the absurdity of Apple's position here. It's simply baffling why one of the largest open-source projects they sponsor--WebKit--violates platform protocols by using private API's, and apps directly bundling such code can't be deployed on the platform's major distribution channel, the Mac App Store. Wouldn't it be better for Apple to open up these private API's, make them public, and allow third-party developers to use them if necessary? WebKit's use of these API's dates back to the earliest days of the project; I found commit messages from 2002 that report their inclusion. Tk would certainly benefit if use of those API's could be made legal. If a platform vendor's private API is used in a vendor-sponsored open-source project, how truly private is the API? If Apple is going to be consistent here, shouldn't WebKit remove these private API calls, and find another way to render browser windows in a smooth, crisp fashion?

Past a certain point, griping is pointless; this is where we are. And I don't want to be too critical of Apple; Apple has greatly benefited the Tcl/Tk community by sponsoring Daniel Steffen's work on the Cocoa port of Tk--I doubt it would have been written otherwise. No other major open-source GUI toolkit was the beneficiary of such largesse. Still, it's a sad situation that complying with Apple's guidelines has resulted in the measurable degradation of the GUI toolkit whose port Apple sponsored. It may be a necessary compromise, but it's hard to be happy about it.

[/general] permanent link

Mon, 15 Sep 2014

Tkmacicon 1.2

I've released version 1.2 of my tkmacicon package for Tcl/Tk on Mac OS X.

The tkmacicon package provides the data necessary to platform-native icons on OS X as Tk images. The ::tkmacicon::geticonfrompath command takes three arguments: a file path, width, and height. The ::tkmacicon::geticonfromtype command takes three arguments: a file extension, width, and height. These commands provide the raw image data necessary to render a Mac icon as a native Tk image.

This release represents a significant API change from earlier versions, which wrote the image data to a PNG file and then read the file into a Tk image. Obtaining the data without the overhead of file writes and reads makes the package significantly faster.

For more information, see http://opensource.codebykevin.com/native.html#tkmacicon.

[/software] permanent link

Progressdock

I've released version 1.0 of my progressdock package for Tk on Mac OS X.

The progressdock package draws a nice progress bar over a Tk/Mac application's Dock icon. It is especially useful for showing download/uploads, and similar activities. The package is based on the UKDockProgressIndicator class by Uli Kusterer.

For more information on the progressdock package, see http://opensource.codebykevin.com/native.html#progressdock.

[/software] permanent link

Sat, 06 Sep 2014

Deployment of Perl and Ruby Tk apps on the Mac: Coming along

Over the past few years I've experimented with both Ruby and Perl as development languages for desktop apps on the Mac. Both are robust programming languages, have superb library support, and have excellent bindings for my GUI toolkit of choice. However, neither language is used much on the Mac for development of desktop apps. Their achilles' heel is deployment; in contrast to Python or Tcl/Tk, neither language has a standard, widely-used mechanism for bundling up code into a single standalone Mac application package that can be run without issue.

I've spent a lot of time over the past year or so investigating the deployment issue, periodically pulling out a couple of simple Ruby or Perl projects that I play with as part of the process of learning these languages as desktop app tools. And I'm happy to report that I've made good progress and have prototype apps running as standalone apps on the Mac.

In keeping with Perl's "there's more than one way to do it" spirit, I've actually found several ways to deploy an app. The one I've been working with the most is the Perl Archiving Toolkit and its pp module, which bundles up Perl code into a standalone executable than can then be run on another system. This works quite well, actually, and one could do the extra work of deploying the executable inside a Mac application bundle structure (a Mac app bundle, though it looks like a single file on the Mac, is actually a special directory structure). However, this mode has limitations, particularly with linking to the Tcl/Tk frameworks. The pp module has no knowledge of Mac application structure and no easy way to use standard Mac mechanisms for finding a Tcl/Tk installation bundled with the application. This proved to be a serious, and surprisingly difficult-to-solve, issue.

As a result, I decided to look elsewhere, and eventually found a much better solution in Perl's library ecosystem: Mac::QuckBundle. The QuickBundle library incorporates yet another library, mac-perl-wrapper, and does two things: it creates a proper Mac application structure for the deployment of Perl code, and automates the process so that an app can be built using a single configuration file. With this structure in place, it is much for me easier to do some additional configuration so that Perl can find the correct Tcl/Tk libraries in the application package.

Mac::QuickBundle is a near-perfect tool for my needs here, quite analogous to py2app or cx_Freeze in the Python world. I'm rather surprised I hadn't heard of it before, but I think this is reflective of the fact that Perl does not have a large constituency in the Mac desktop development community. Still, I'm glad the code was out there.

Ruby has proven to be a much more complex language to work with. Despite all my research, I've never found a general-purpose tool for deploying Ruby apps on the Mac. What tools exist are either Windows-only or limited to deep Ruby/Cocoa integration, which doesn't meet my needs. Still, after more work with Ruby and studying how the Perl and Python apps work, I've put together the basic steps to deploy a Ruby/Tk app on the Mac:

  1. Create a small stub launcher in C that embeds Ruby. That seems to be the secret sauce to nearly all the tools I've studied; they start as small applications written in C that then launch an embedded interpreter. This isn't surprising, actually, and simplifies the basic structure of the application package.
  2. Use MacPorts to build Ruby and build Ruby with an "--enable-load-relative" flag. There are dozens of ways to build Ruby on the Mac; it is actually not easy to do. Of all the ways I've tried, MacPorts seems to have the simplest, most complete support for building Ruby. I submitted a patch to MacPorts to allow Ruby to build under a "load-relative" flag, which means that its search path for its core libraries is not hard-coded (the default mode) but rather is encoded to be relative to the main Ruby executable. Building "load-relative" means that the Ruby installation can be relocated to another location, such as an application bundle, and will still work as long as the overall installation structure is preserved. (Otherwise the deployed Ruby will look in a fixed location, such as /opt/local, for all of its supporting code, which won't work if nothing is installed there.)
  3. Use MacPorts's search mechanisms to find the locations of the appropriate libraries and copy them into the application bundle. Ruby does not seem to have any mechanisms for searching and identifying code that an application depends on (Perl has its "scandeps" library and Python has its "modulefinder" library). However, MacPorts itself provides tools for searching out its installations, such as pkg-config, and these tools make it possible to build and link the application launcher, and find and copy over the Ruby installation, seamlessly.

Right now my work with Ruby has been distilled into a shell script that creates my builds; I'm not sure if it can be generalized into a general-purpose application builder for Ruby. Ruby still has too many quirks to be as easily deployable as Python or Tcl/Tk, for instance. But I will likely post the script and some documentation at some point in the future, when it's been further tested and I have a releasable app to use with it.

I'm pleased that I've been able to make progress with deployment of Perl and Ruby apps on the Mac; this work will allow me to access the considerable power of Perl and Ruby for new desktop applications.

[/software] permanent link

Sparkle back in business

A few months ago I replaced the popular Sparkle update mechanism for the Mac in my apps with a hand-rolled mechanism that used some of Sparkle's ideas and data format, but which was simpler to configure in my own code. This was motivated by the fact that Sparkle seemed to have fallen into obsolescence, having been unmaintained for a few years.

I'm pleased to report that development on Sparkle has resumed, with a new project page. Sparkle now has a community of developers working on it, and it seems to have received several much-needed updates. While I don't plan to go back to using Sparkle in my own programs, I'm very pleased to see that this vital tool show signs of life on the Mac.

[/software] permanent link