Wing IDE Professional

Introduction to Features and Implementation

The presentation starts on the next slide.

To adjust font size use ctrl +/- or command +/- on OS X; arrow keys and space bar move through the slides; mouse over lower right to view the slides with notes or jump to a page.

This presentation was developed with S5

Wing IDE Professional

Introduction to Features and Implementation

Stephan Deibel and John Ehresman

Wingware

About This Talk

This talk has three parts: A demo of Wing IDE Pro 3.0 beta3, a presentation on our experiences developing Wing, and hopefully a sizeable chunk of time where we can talk more about topics of interest or help people try out Wing.

We've tried to focus on things that would be useful to other programmers thinking about starting a small Python-based company, or just things that I think help us keep productive and focused. Since we're an IDE company, some of this revolves around the IDE and what we think makes the IDE worth using as a productivity tool (from our own experiences with). In IT Speak, all of this might be called "best practices".

If you want to try things out during the demo, please download Wing and run it.

About Wingware

Wingware was founded by John Ehresman and Stephan Deibel in 1999. It was originally Archaeopteryx Software, Inc until it became obvious no one could (nor wanted to learn) how to spell "Archaeopteryx".

We have basically one product, with three product levels. Wing Pro is the "flagship" product with the full feature set we'll demo today. Personal is a much cheaper subset of Wing Pro. Wing 101 is a recent addition -- a heavily scaled back free IDE designed with the University of Toronto to teach entry level computer science courses.

About Wing IDE

The core product is about 7 years old now. We saw a fairly typical 3-5 year startup period before becoming profitable.

The statistics given here are as of August 2007, and vary a bit by month.

Usually, 50% of sales are outside the US, with customers (or at least users) in every country in the world that has some kind of software development industry. We've been known to provide support in languages we don't even speak, using Babelfish. We do have a German and partial French localization but not very many people seem to use those. Programmers are used to thinking and speaking in English when programming. The cost of maintaining localizations is somewhat prohibitive; one was done by an intern and the other by one of our customers and contributed back.

About 2/3 of our users are on Windows, with 30% on Linux, and the rest on OS X and others. People can get the source code and self-compile Wing, on Solaris, FreeBSD, the PlayStation3, etc.

Wing is, possibly, the best-selling or "leading" commercial Python IDE. We don't have hard statistics on this and don't really know what our market share is. However, we're the only company actively developing a commercial IDE that targets specifically Python, and we've put quite a few years of development into it.

Wing IDE Demonstration



If you're reading this as a handout, you may want to try out Wing now by using the Tutorial from the Help menu. This shows the basic stuff. The demonstration we're doing live will cover a bit more, including the following:

  * Overall layout: Panels, editor, show/hide, tools menu
  * Tutorial in Help menu
  * Prefs:  Keyboard personality, tab key, windowing mode, many others
  * Setting up project:  Add directory, Project Properties (python,
    environment, and revision control)
  * Editing
    * Syntax highlighting
    * Auto-completion, source assistant
    * Index menus, browser, goto definition
    * Evaluate in shell
    * Search interfaces
  * Testing
    * Configuring tests
    * Running tests
    * Debugging tests
  * Debugging
    * Breakpoints (setting, types, breakpoint tool)
    * Starting
    * Looking around:  Stack data, call stack, watch tools
    * Debug Probe
    * Launching process from outside Wing
  * Show documentation & point out How-Tos for many things
  * Other stuff (show briefly or just mention)
    * OS Commands
    * Bookmarks
    * Indentation tool
    * Extension scripting (e.g. pylint)
    * Zope/Plone

See also this Intro to Wing IDE video (although it covers a much smaller subset of features than the tutorial):

http://showmedo.com/videos/video?name=pythonOzsvaldWingIDEIntro

Developing Wing

Wing is written in Python and C. Writing, testing, and debugging Wing with itself is an important part of our quality control strategy. Self-use is one of the reasons open source tends to be of high quality. This should almost be a requirement for all programmers.

The line counts here were done using sloccount, which counts physical lines of code omitting blank lines and comments. As you can see, about 80% of the code base is in Python, the rest in C. Luckily, we spend almost all of our time working in Python as the C parts are fairly mature internals. An exception was during our recent rewrite of the debugger to support threaded debugging -- much of the debugger core is in C.

The person-year and cost estimates are wacky, and a totally unscientific show of The Power of Python. The numbers are however somewhat supportive of anecdotal evidence that Python is up to 10X more productive than C.

Based on Open Source

We use and contribute to a lot of open source. What we build and ship with Wing absolutely dwarfs our own code base, at about 1.7 million lines of source.

We've put perhaps a year full time into GTK and PyGTK, months into Scintilla, and for the rest we are mostly just users. See Wing's About box and manual for a complete list of the modules, authors, and license information.

As you'll see in the About box, we check over and catalog the licenses involved very carefully to make sure we don't violate how the authors have licensed their code. We can't use anything GPL because of the viral terms of the license, but LGPL is fine as well as MIT, BSD, and most others.

In general philosophical terms, we're obviously not purists. We think open source and closed source and and should co-exist. However, we're also very enthusiastic about open source and it's safe to say Wing wouldn't exist without it.

Choosing GTK and PyGTK

Results with GTK and PyGTK

So as it turned out, GTK is really good at "write once works anywhere" and by virtue of PyGTK and Python this even means we can distribute identical Python object code for the GUI in a cross-platform way (it runs against GTK compiled for each platform, of course). We've put significant time into GTK but it was a worthwhile investment that makes our day to day development much easier.

We also like GTK's rich signal model, which makes it very easy to hook functionality up to the GUI. It's also the basis for how C is used to write "classes", which via PyGTK can easily be descended from and overridden in Python. GTK has at its core a decent asynchronous task framework that's used to service the GUI and also can be used to simultaneously service network connections or run periodic tasks. We make heavy use of this, as Wing is entirely based on the asyncronous model rather than threading (more on this later).

Overall GTK and PyGTK's bindings are designed in a way that makes it quite easy to write GUIs either by hand in code, or to write code that builds GUIs based on models. We don't use a GUI builder but we do use some tools we developed to generate parts of the GUI automatically from data models and GUI hints. For example, Wing's preferences and project properties are generated that way. We could talk more about that in the Q&A if there is interest.

Things we don't like about GTK include that it's very hard to build and ship GTK, but we do need to ship it with the app even on Linux because binary compatibility between GTK versions is sometimes an issue. We try very hard to make Wing "just work" (for the customer, and for keeping support manageable) and shipping with GTK included reduces the risk of environment-specific problems. On Linux, --system-gtk on the command line can be used to override our internal GTK and use the system-provided one instead, and this often works.

Another significant issue is that GTK isn't native on OS X and requires an X11 Server. It works pretty well but native would be better. There's work on this and Gimp (also a GTK app) runs native on OS X. However, we guestimate it would take us a year of effort to get it working well. Wing uses more and different GTK functionality from other GTK apps, as we found out on Windows where lots of bug fixing was needed.

GTK also isn't really entirely native on Windows, but it's very close now and complaints have become relatively rare.

Approach to optimization

I've been trying to highlight how we do things in light of our critical need to save time. We learned the hard way how best to write code that is "fast enough" in Python. Initially, we wrote any module we thought needed to be super-fast (like the debugger core and source code analysis engine) entirely in C. This was a mistake. Instead, we've learned to write in Python, optimize only what the profiler (hotshot) reveals as slow, and try to do that only in Python. As a last resort, we write small bits in C. We managed to do that with the search capability, some file I/O code, and some object life cycle stuff that ended up being performance critical. It's much easier to maintain than the pure C code. Again, anecdotally, you'll get 10X as much done in Python.

We've not used pyrex though it looks like a good solution. I suppose we're just old salts and writing a few bits of extension module is too easy... although this is an area where we may be kidding ourselves and are not working as efficiently as we could.

Psyco is a bit problematic. It crashes Wing when in full optimization mode which seemed to indicate that it might be a crap shoot when in profile-guided mode. So we're afraid of it, as a potential cause of instability. This is important. Programmers really don't like unstable tools -- they have enough to do making their own code work properly. And it would cost us support and debugging time.

Async not threads

Wing uses asynchronous tasks rather than threads to get its work done. We split up background tasks like source analysis into small units that run interleaved with the event loop. This is a big time saver for us. The time we spend splitting up tasks into small units is much less than the time it would take to track down complex threading bugs. Async is generally easier to manage because the context switch is much more predictable.

With generators it's really easy to do break a loop into units as in the trivial example above, and hook these into the GTK mainloop.

We're not using Twisted because GTK provides what we need, so it may not provide much added value in our case. But we hear it is wicked cool for other things.

Unit testing (and debugging)

We write unit tests for Wing, tending to focus particularly on code that is hard to get right, or even more importantly on code that is hard to keep right when further modifications are made. We try to avoid writing tests for things we'ld immediately notice from our self-use of the product.

We haven't bought into test-driven development as a panacea, but it's an extremely important tool that every programmer should be aware of an use according to the nature of their project.

Another thing we don't agree with is the assertion sometimes seen that somehow unit testing obviates the need for the debugger. We believe strongly that a debugger has an important place even in test-driven development. It's much faster than writing print statements or reading through tracebacks and manually going up and down the stack to find a problem. If a unit test fails, we run it in the debugger to save time.

Debuggers: not just for bugs

The debugger also works well as a code learning tool, even when no bugs are present. It's the fastest way to understand how code is being invoked, what datastructures actually contain at runtime, and how dynamic or tricky code works. We often hear from people using Wing's debugger along with the source browser to decipher Zope's internals.

To add a new feature, it's often easiest to place a breakpoint in the code to be edited, run to the breakpoint, and design the new code in Wing's Debug Probe. The auto-completer and source assistant, which key off actual runtime state in this case, will make it easier to design the correct code quickly, even in unfamiliar territory or in complex code, and then try it right away.

Customer-driven Development

When we started out, we thought as programmers that we had a pretty good handle on the nature of the IDE market. We soon found out that even programmers don't know what other programmers really want in an IDE.

We hear strange (to us) or directly contradictory feedback almost daily, sometimes sitting in adjacent emails in our inboxes (see above). Suffice it to say that programmers have a very diverse set of ideas of what an IDE should be. Some of this depends on what other IDEs they have used, some derives from whether they are a hobbyist or a consultant or an employee, and of course personality comes into play as well. The IDE is a much more intensely personal tool that we originally understood it to be.

Once we figured out that we can't see Wing from the same perspective as customers, we shifted the focus of our development to being much more customer requests driven. Putting the customer first both makes for a better more appealing product and has clearly reduced the amount of support we need to provide. Not surprisingly, if you listen to the customer, things work more often like the customer expects.

Feedback, bug reports, and patches

One aspect of being more customer-driven was to add feedback and bug report functionality to the product. Not surprisingly this made things work, more often, like the customer expects and we started receiving fewer duplicates of similar reports and requests. We also stopped spending a lot of time on bugs or quirks that we found users would never encounter in the field, and more importantly we fixed the ones people did run into all the time (even if we didn't).

When we do hear from customers, we try to respond with an hour, sometimes faster. This keeps the customer engaged moving on their evaluation or in their work, rather than putting Wing aside or getting frustrated. It also means that we can get more details on bugs they have encountered before the lose the context in which they found the bug. Being responsive makes a huge difference and we hear a lot of positive feedback about our support, because clearly this isn't typical in the industry. It costs us some effort and time, but we offset that by fixing problems sooner so we get fewer repeat reports.

Wing comes with a patch manager that lets us respond very quickly to critical bugs: We can fix them and patch them sometimes within an hour then email the customer to "Check for Updates" from Wing's Help menu. This downloads any updates and restarts Wing with the fix in place. Quick fixes also mean we don't have to answer the same questions over and over again, so this saves us a lot of time, and improves customer satisfaction.

Custom build and packaging

We wrote our own build system and packager for Wing, which is used to build all the open source we depend on, and Wing itself. We don't use py2exe and pyinstaller, partially because we predate adequate versions of those tools and partly because we have to deliver the debugger support modules for so many different versions and build variants and implementations of Python (which is somewhat of a combinatorial explosion at this point). Wing is delivered as a native installer on win32, RPM or Debian or tar package on Linux, and as a disk image on OS X.

At this point, all we have to say is "./build all-releases" on four machines, and run a few additional tasks and scripts and we've got a release on our website. While this runs, I can usually start working on some more bug fixes or features for the following release.

Review of key enablers

Here's an review of the technical things I attribute our success to, at least to the extent that we can support three products on three OSes to a very diverse customer base, all while still moving the product ahead.


Q&A


Thanks for Coming!


Stephan Deibel and John Ehresman

Wingware | wingware.com

For more information, please don't hesitate to contact us at support@wingware.com.