Skip navigation
Help

XML

warning: Creating default object from empty value in /var/www/vhosts/sayforward.com/subdomains/recorder/httpdocs/modules/taxonomy/taxonomy.pages.inc on line 33.

I suppose What You See Is What You Get has its place, but as an OCD addled programmer, I have a problem with WYSIWYG as a one size fits all solution. Whether it's invisible white space, or invisible formatting tags, it's been my experience that forcing people to work with invisible things they cannot directly control … inevitably backfires. A lot.

I have a distinctly Ghostbusters attitude to this problem.

Ghostbusters-logo

I need to see these invisible things, so that I can zap them with my proton pack. I mean, er, control them. And more importantly, understand them; perhaps even master them.

I recently had the great privilege of meeting Ted Nelson, who gave me an in-person demo of his ZigZag project and his perpetually in-progress since 1960 Xanadu project, currently known as Xanadu Space. But one thing he mentioned as he gave the demo particularly intrigued me. Being Ted Nelson, of course he went much further than my natural aversion to invisible, hidden markup in content – he insisted that markup and content should never be in the same document. Far more radical.

I want to discuss what I consider one of the worst mistakes of the current software world, embedded markup; which is, regrettably, the heart of such current standards as SGML and HTML. (There are many other embedded markup systems; an interesting one is RTF. But I will concentrate on the SGML-HTML theology because of its claims and fervor.)

There is no one reason this approach is wrong; I believe it is wrong in almost every respect.

I propose a three-layer model:

  1. A content layer to facilitate editing, content linking, and transclusion management.
  2. A structure layer, declarable separately. Users should be able to specify entities, connections and co-presence logic, defined independently of appearance or size or contents; as well as overlay correspondence, links, transclusions, and "hoses" for movable content.
  3. A special-effects-and-primping layer should allow the declaration of ever-so-many fonts, format blocks, fanfares, and whizbangs, and their assignment to what's in the content and structure layers.

It's an interesting, albeit extremely hand-wavy and complex, alternative. I'm unclear how you would keep the structure layer in sync with the content layer if someone is editing the content. I don't even know if there are any real world examples of this three layer approach in action. (And as usual, feel free to correct me in the comments if I've missed anything!)

Instead, what we do have are existing, traditional methods of intermixing content and markup ala HTML or TeX.

PDF vs. TeX

When editing, there are two possible interfaces:

  1. WYSIWYG where the markup layer is magically hidden so, at least in theory, the user doesn't ever have to know about markup and can focus entirely on the content. It is an illusion, but it is simple enough when it's working. The downside is that the abstraction – this idea that the markup is truly "invisible" – is rarely achieved in practice and often breaks down for anything except the most basic of documents. But it can be good enough in a lot of circumstances.
  2. Two windows where the markup is fully visible in one window, and shown as a live rendered preview in the other window, updated as you type, either side-by-side or top-and-bottom. Users have a dynamic sandbox where they can experiment and learn how markup and content interact in the real world, rather than having it all swept under the rug. Ultimately, this results in less confusion for intermediate and advanced users. That's why I'm particularly fond of this approach, and it is what we use on Stack Exchange. The downside is that it's a bit more complex, depending on whether or not you use humane markup, and it certainly takes a bit more screen space and thinking to process what's going on.

What I didn't realize is that there's actually a third editing option: keep the markup visible, and switch rapidly back and forth between the markup and rendered view with a single keystroke. That's what the Gliimpse project reveals:

Please watch the video. The nearly instantaneous and smooth transition that Gliimpse demonstrates between markup and preview has to be seen to be appreciated. The effect is a bit like Expose on the Mac, or Switcher on PC. I'm not sure how I feel about this, mainly because I don't know of any existing IDEs that even attempt to do anything remotely like it.

But I'd sure like to try it. As a software developer, it's incredibly frustrating to me that we have generational improvements in games like Skyrim and Battlefield 3 that render vastly detailed, dynamic worlds at 60 frames per second, yet our source code editors are advancing only in tiny incremental steps, year after year.

0
Your rating: None

The future is flexible, and we’re bending with it. From responsive web design to futurefriend.ly thinking, we’re moving quickly toward a web that’s more fluid, less fixed, and more easily accessed on a multitude of devices. As we embrace this shift, we need to relinquish control of our content as well, setting it free from the boundaries of a traditional web page to flow as needed through varied displays and contexts. Most conversations about structured content dive headfirst into the technical bits: XML, DITA, microdata, RDF. But structure isn’t just about metadata and markup; it’s what that metadata and markup mean. Sara Wachter-Boettcher shares a framework for making smart decisions about our content's structure.

0
Your rating: None

CityGML (City Geography Markup Language) is an XML-based schema for the modelling and exchange of georeferenced 3D city and landscape models that is quickly being adopted on an international level.

libcitygml is a small and easy to use open source C++ library for parsing CityGML files in such a way that data can be easily exploited by 3D rendering applications (geometry data are tesselated and optimized for rendering during parsing). For instance, it can be used to develop readers of CityGML files in many 3D based applications (OpenGL, OpenSceneGraph, ...) Most metadata are not lost, they are available through a per-node hashmap.

libcitygml is developed by the 3D team of BRGM (the French leading public institution involved in the Earth Science field for the sustainable management of natural resources and surface and subsurface risks) for the research project DeepCity3D.

The following screenshots were done using the CityGML reader implemented for OpenSceneGraph. The models used are freely available on the official CityGML site.

The following screenshots are the conversion to VRML97 of the examples of CityGML Tunnel and Bridge ADEs:

Supported features

libcitygml is not a strict implementation of the CityGML standard and is not based on a particular schema. However is has been tested successfully with CityGML version 0.3 to 1.0 files, all LOD (from 0 to 4) and supports most of the model features including:

  • Buildings and building furnitures
  • Digital Terrain Models as triangulated irregular networks (TINs)
  • Vegetation (areas, volumes and solitary objects)
  • Water bodies (volumes, surfaces)
  • Transportation facilities (both graph structures and 3D surface data) like roads or railways
  • City furnitures
  • Tunnels (subsurface structures) & Bridges

Materials & textures are also well supported.

0
Your rating: None

I recently did a complete rewrite of my graph-based A* pathfinder example because I received a lot of questions on how to implement path-finding using the new ds library. So here is the updated version which works with ds 1.32:

I’m transforming the point cloud with delaunay triangulation into a graph structure. Then the system computes and draws the shortest path between two selected points.

Compile instructions

Running and examining the example is really easy:

  1. Use the automated installer to install haXe from http://haxe.org/download.
  2. Download the latest haXe nightly build and overwrite the existing ‘haxe.exe’ and ‘std’ folder with the downloaded version.
  3. Install the polygonal library by opening the command prompt and typing:
    haxelib install polygonal.

Sources should now be in {haxe_install_dir}/lib/polygonal/1,18/src/impl/sandbox/ds/astar, where {haxe_install_dir} is usually C:/Motion-Twin/haxe on Win7.
The demo can be compiled with:
cd C:\Motion-Twin\haxe\lib\polygonal\1,18\build
haxe.exe compile-ds-examples.hxml

Extending the Graph class

You have basically two options to extend the functionality of the Graph object: by composition or inheritance. While I highly recommend to use composition whenever possible, I’ve also included a version using inheritance – just so you see the difference.

The composition version looks like this:
astar using composition
The Graph object manages GraphNode objects, and each GraphNode holds a Waypoint object, which defines the world position of the waypoint as well as intermediate data used by the A* algorithm. Notice that GraphNode and Waypoint are cross-referencing each other as a Waypoint object has to query the graph for adjacent nodes. As a result, you have a clean separation between the data structure (Graph, GraphNode) and the algorithm (AStar, Waypoint) and don’t need object casting, which is good news because casting is a rather slow operation.

Now let’s look at the version using inheritance:
astar using inheritance
Here, Waypoint directly subclasses GraphNode. Since the Graph is defined to work with GraphNode objects, we need a lot of (unsafe) down-casts to access the Waypoint class. Furthermore, the use of haxe.rtti.Generic will be very restricted or even impossible (implementing this marker interface generates unique classes for each type to avoiding dynamic).

3
Your rating: None Average: 3 (1 vote)

Download the eXtremeDB data sheet (PDF)

Explore the key eXtremeDB features that enable developers to create the most advanced software applications using McObject's real-time database technology.

The eXtremeDB™ In-Memory Database System embedded database is McObject's core product: an exceptionally fast database management system, designed for performance, with a strict memory-based architecture and direct data manipulation. (For a database that incorporates on-disk persistence, see McObject's eXtremeDB Fusion product.) Storing and manipulating data in exactly the form used by the application removes overheads associated with caching and translation. Typical read and write accesses are at the level of a few microseconds, or less. The engine is reentrant, allowing for multiple execution threads, with transactions supporting the ACID properties, assuring transaction and data integrity.

The Runtime Environment

The eXtremeDB Runtime Environment provides:

  • Accelerated transactions. The eXtremeDB in-memory database stores data entirely in main memory, eliminating the need for disk access, caching and other processes that add overhead to disk-based databases. The eXtremeDB transaction manager is optimized for high transaction rates.
  • Ultra-small footprint. By intelligently redesigning and streamlining core database functions, McObject offers an in-memory database system (IMDS) with an unbelievably small RAM footprint of approximately 100K! This makes eXtremeDB a powerful enhancement to many intelligent devices with resource limits that, until now, ruled out the use of an embedded database system.
  • Direct data access. Earlier data management technology required copying records from database storage to cache, and then to a new location for manipulation by the application. By operating as a RAM database that works with data directly in main memory, eXtremeDB eliminates the overhead of duplicate data sets and of copying data between locations.
  • No Translation. eXtremeDB stores data in the exact form in which it is used by the application - no mapping a C data element to a relational representation, for example, or requiring additional code to pick fields from tables and copy them to C structures. By eliminating this overhead, eXtremeDB reduces memory and CPU demands.
  • High reliability. For data integrity, eXtremeDB transactions support the ACID properties, ensuring that operations grouped into transactions will complete together or the database will be rolled back to a pre-transaction state.
  • eXtremeDB provides two APIs. The first is a standard function library for basic operations like cursor movement and opening and closing the database. The second API, for manipulating data, derives from the given application’s data model and thus reflects the purpose and schema for which it is being used. For the runtime environment, this means more reliable code – the C/C++ compiler will catch data typing and assignment errors when the application is built. This makes eXtremeDB-based applications more reliable, since it is much harder for coding errors to make it into the final build.

eXtremeDB is designed for High-Performance: optimized memory managers,  hash and tree-based indices, multiple data layouts, transactions with priority scheduling and an application-specific API.

 

Shared Memory Databases

In addition to the eXtremeDB embedded database that operates in conventional memory, a shared memory version is available for multi-processing environments such as Solaris, QNX or Linux. With this version, an eXtremeDB database is created in shared memory and mapped to the local address space of each process, thereby allowing multiple processes and multiple threads within each process to share eXtremeDB in-memory databases.

The shared memory eXtremeDB runtime is built as a different binary (library or archive) than the conventional memory version. A single process can still create and connect to the database from multiple threads using the shared memory runtime, however, the database(s) will be placed into shared memory instead of the process’ memory. Depending on the target platform, eXtremeDB supports one of the following three synchronization methods when managing shared memory databases:

  • A System V semaphore mechanism is implemented for operating environments such as Sun Solaris and Linux platforms (System V semaphore methods are associated with system-wide integers, called keys, which are associated with files).

  • A POSIX shared memory implementation that is suitable for QNX 4.x and QNX 6.x platforms.

  • A Win32 shared memory implementation is deployed for the Microsoft Windows Embedded and Microsoft Windows classic platforms.

 eXtremeDB XML Extensions

McObject developed the eXtremeDB XML Extensions to facilitate simple schema evolution and the exchange of data between the eXtremeDB embedded database and external systems. With the XML-enabled version, the eXtremeDB schema compiler generates new interface functions for each object that provide the means to

  • retrieve an object encoded as XML
  • create an object in the database from an XML document.
  • replace (update) the contents of an object already in the database with the content of an XML document
  • generate the XML schema for each class in the database.

The XML interface functions could be used, for instance, in concert with the eXtremeDB event notifications to cause data to be shared between an eXtremeDB embedded database and other XML-enabled systems when something of interest changes in the database. The XML interfaces can also be used to facilitate simple schema evolution by exporting the database to XML documents, adding/dropping fields, indexes, and classes, and importing the saved XML documents into the new database.

eXtremeDB XML and eXtremeDB XML schema encoding were developed in accordance with the W3C SOAP encoding recommendations. Thus, the XML interface functions could also be used in conjunction with an embedded Web server to deliver an embedded systems database context (from an application running within a consumer electronics device, for example) to a Web browser or any other SOAP client. The picture below illustrates this process.

SOAP Standards

W3C SOAP encoding recommendations can be found in the following documents:

http://www.w3.org/TR/soap12-part0/
http://www.w3.org/TR/soap12-part1/
http://www.w3.org/TR/soap12-part2/

XML schema encoding recommendations can be found in the follow W3C documents:

http://www.w3.org/TR/xmlschema-0/
http://www.w3.org/TR/xmlschema-1/

 The Development Environment

Developers strive to produce readable, maintainable, efficient code in the shortest possible time. The eXtremeDB in-memory database system (IMDS) includes several features that boost the developer’s capabilities when integrating the database in demanding real-time applications. Incorporating third party software often means learning and adopting an API that does not completely fit an application. eXtremeDB’s project-specific API ensures that each database operation in the API reflects the type of the data being manipulated.

To help in application debugging, McObject’s embedded database includes numerous traps in the runtime code; these can be selectively disabled as development and testing progresses, to optimize the application for speed.

McObject offers full source code, to give an in-depth understanding of eXtremeDB within an application. In addition, eXtremeDB supports virtually all data types as well as extremely efficient indexing for queries. For querying, McObject provides hash indexes for exact match searches; tree indexes for pattern match, range retrieval and sorting; and object-identifier references, for direct access. Rather than storing duplicate data, indexes contain only a reference to data, keeping memory requirements for the RAM database to an absolute minimum.

With eXtremeDB embedded database, the developer focuses on the data definition first, then eXtremeDB generates the API from this definition via the schema compiler.

The result is:

  • An easy-to-learn API that is optimized for the application.

  • Code that is more legible as well as easier to write and maintain.

  • Compile-time type checking that helps eliminate coding errors.

Example: The following is a (simple) class and an example of the API to put a new value into a record in the database:

class Measurement {
  string measure;
  time timestamp;

  unique tree <measure, timestamp> trend;
};

Measurement_measure_put(&m, meas);
Measurement_timestamp_put(&m, value);

Progressive error detection and consistency checking features

If an application mistakenly passes a corrupted transaction or object handle into a runtime method, eXtremeDB (by default) raises a fatal exception and stops the execution of the program. In most cases, the application developer can then withdraw and examine the call stack to find the source of corruption. The eXtremeDB runtime implements many verification traps and consistency checks. Obviously, that does not come free; the runtime requires extra CPU cycles and space for that. However, when the application is debugged and consistently passes verification tests, developers can generate the optimized version of the eXtremeDB runtime, removing the traps and internal checks to restore valuable clock cycles.

Complex data types and efficient queries

  • Supports virtually all data types, including structures, arrays, vectors and BLOBs

  • Querying methods include hash indexes for exact match searches

  • Tree indexes support queries for pattern match, range retrieval and sorting

  • “Voluntary” indexes for program control over index population

  • Object-identifier references provide direct data access

  • Autoid for system-defined object identifiers

  • Rather than storing duplicate data, indexes contain only a reference to data, minimizing memory requirements

  • Synchronous/asynchronous event notifications

  • Optional object history

Supported Platforms

Embedded Platforms:

  • VxWorks
  • QNX Neutrino
  • Various Real-Time Linux distributions
  • Lynx OS
  • RTXC Quadros, RTXC
  • INTEGRITY OS
  • Microsoft Windows Embedded Platforms
  • Windows Real-Time Extensions
  • Bare bones boards (no operating system required)

Server and Desktop Platforms:

  • Sun Solaris
  • HP-UX
  • Linux distributions
  • Classic Windows platforms (98/NT/2000/XP/Vista)

 Development Environments:

  • gnu toolchain (gcc 2.95 and higher)
  • Tornado (GNU and Diab compilers)
  • QNX Momentics IDE (C, C++, Embedded C++)
  • Metrowerks CodeWarrior IDE (various platforms)
  • GreenHills Multi
  • Microsoft Visual Studio (C/C++)
0
Your rating: None

I’ve just released the first official version of ‘ds‘ (aka data structures), the successor of as3ds which is written in the HaXe language. (The name hx3ds was used during development and is obsolete because ds is now a module of the polygonal library).

The library is found here: http://code.google.com/p/polygonal/. There is also wiki page describing how to use it with ActionScript 3.0 (there are still some issues with the HaXe generated SWC files, I hope Nicolas finds some time to address them in a future release).

What’s new

The new release (changelog) contains many bug fixes, new features and a refined documentation. While there are still many things on my TODO list, I think the library is stable enough so it deserves to be the first major release. And since I’m using the library in all of my projects, you can expect some regular updates.

Why the heck HaXe !?

Firstly, ActionScript 3.0 doesn’t support any kind of generics/templates like Java does for example. The typed vectors introduced in FP10 are too crippled and limited so in the end you are still stuck with dynamic containers, which is a bad for complex projects. HaXe supports type parameters – in the image below you see that I’ve created a graph of stacks of hash maps and everything is typed. The nice thing is that you still get auto-completion:

FlashDevelop HaXe AutoCompletion

Secondly, HaXe provides a lot of syntax sugar that sweetens your daily coding experience. As an example, consider iterating over a collection. In HaXe it’s just:

//iterate over all elements of a 2D array:
var a = new Array2<Int>(3, 3);
for (i in a) trace(i);

In AS3 you have to create an iterator object and explicitly call the next() and hasNext() methods:

var a:Array2 = new Array2(3, 3);
var itr:Object = a.iterator();
while (itr.hasNext())
{
  var element:* = itr.next();
  trace(element);
}

In general, HaXe implicitly does a lot of things for you! Probably one of the best language feature is type inference/implicit typing, which is absolutely brilliant for quick prototyping. It’s like writing AS1 but without compromising speed and type safety. At first it’s hard to get used to it because all ActionScript text books never get tired of repeating how important typing is and that everything should be typed. But If you think about it, the compiler should handle it where possible and relieve the developer from writing clumsy types over and over again:

//AS3
var n:Number = 1;
var i:int = 1;

//HaXe
var f = 1.0; //compiler infers that n is a float
var i = 1; //compiler infers that n is an int

Thirdly, because as3ds was all about efficiency I couldn’t resist HaXe because it’s much faster. From my experience, ds performs ~2-6x better than as3ds. Same runtime, huge difference!

0
Your rating: None

Two Directory Source Control is a work flow where you utilize one directory from source control, one directory that you work out of, and you then shuttle between them at your convenience. In this article, I’ll describe how traditional work flows work, the problems with them, and how you can use the Two Directory option to overcome them. This article assumes you know what source control is, what a merging tool is, and the value they bring to your work flow.

Traditional Source Control Work Flows

Many tools such as Flex Builder , Flash , and Visual Studio have plug ins that allow integration with source control such as Subversion , CVS , Perforce , and Visual Source Safe . This allows you to check out code & assets, edit them, and check your changes back into source control directly from your IDE of choice. Tools such as Subclipse , a plug in for Eclipse , allow you to check out & check in code from an SVN repo directly into your project. It also has nice built-in "diffing", the ability to show 2 pieces of code side by side to allow you to see what’s changed, and merge them if you so desire.

The reality, however, is that these work flows aren’t as easy as the marketing makes it sound. The main flaw they all have is conflict resolution.

Conflict Resolution

Conflicts are when the code you have on your machine is different from what is in source control. Whether this is because the file in source control is newer, someone has the file checked out already in Visual Source Safe, or a folder/package structure has been re-arranged making your structure out of sync. My main gripe with the tools I’ve used is that they do not do an adequate job of making conflict resolution easy. I define easy as simple to understand what the conflict is, how to resolve it, and not interrupting your current work forcing you to immediately deal with the conflict. To expound on that last part, I agree that you must deal with conflicts when checking in code, but a conflict should not immediately break your current build until you resolve a conflict. Nor should it prevent you from working.

There are 2 scenarios where this irritation is shown. The first example is where you check out the latest code from SVN for your project. You then make your Flex Project in that directory. Assuming you’ve figured out how to prevent Tortoise SVN from copying .svn folders into your bin directories, you download the latest, and get to work. How long you work, however, is based on how often your team checks in code. Even if you are working on different portions of the code, if someone checks in code that relates to yours, you’ll have to deal with the conflict. Typically, you’ll do an update first, and once you’ve ensure none of the code you are committing needs to be merged via resolved conflicts, you commit your changes.

As your team size grows, however, many people will be checking in code. Since dealing with conflicts isn’t fun, sometimes teams will check in more often. This results in a behavior re-enforcement for others to check in sooner as well. While I’m a firm believer in having something compiling sooner than later is a great Agile practice to follow to reduce risk, it’s hard to really make head way on some of your code if your focused on getting code compiling quicker so you can check in vs. making progress on your code.

It may seem subtle, but it’s a major attitude adjustment that I perceive as harmful for developers to have. They should enjoy coding their masterpieces, and assuming they are communicating with others and/or their team lead, they should be focusing. A developers focus is a very important commodity that shouldn’t be taken lightly. Instead of coding by positive reinforcement, "I can’t wait to check this in for others to see!", they are instead coding by negative reinforcement, "I need to hurry up and check in what I have working so I don’t have to deal with a confusing merge operationthat’ll slow me and my team down".

Eclipse combined with Subclipse has some good merging and history management tools. Eclipse has it’s own local history where it stores multiple versions of your files. You can choose to revert if you need to and it’ll provide a diffing tool right there for you. My first problem is the diffing tool sucks. Subclipse is not much better. I last used it in February of 2007; if it has improved since then, let me know. This wouldn’t be a big deal, except when it’s 5:59 pm on a Friday evening, and you really really want to go home. Unless you resolve the conflict, your build will be broken, and you cannot commit till you do so. Suddenly, a good diff tool becomes very important. It’s really frustrating for a developer, and removes motivation to work later. To compound it, most developers I know refuse to leave until their code is compiling. Suddenly an update interferes with this.

The second scenario which is worse than the above is package re-factoring. This is when you change the directory of a file, or a whole folder structure. If it is minute enough, you can usually get by if its not a directory you were working in. If, however, it is AND files have change, this can be a really long and frustrating process… all the while your code no longer works. You cannot compile JUST because you wanted to do the right thing and check your code in. This is also where you can truly tell if a diffing tool is good or not; how it helps you resolve folder changes and their relations to files.

One solution to this is to check your code into a branch and merge in later when it is convenient for you.

The last problem with working directly from the repo is actually caused by a feature of a lot of source control applications. This is the ability to only check in a directory with files that you are working on. For example, if your team is using Cairngorm, and all you are doing isValueObject and Factory work, you can pretty much stay in those 2 directories, and not worry about someone stepping on your toes.

…it is a false sense of security, however. The cardinal rule for source control is only check in code that compiles. This doesn’t mean that it compiles in your machine. What this means is if you were to check out the entire build from source control, could you get to compile? By updating just your area of interest from source control, and then committing your changes successfully in that area, in this case "vo" and "factories" packages, does not mean you have followed this rule. If another developer working on the Cairngorm "delegates" was using your Factories, her/his code would break even though you successfully checked code in. In effect, you broke the build. At one of my older jobs, whoever broke the build had to change their IM icon to a monkey. It sounds trivial, but it really sucked when people asked you why your IM icon was a monkey…

To prevent this, you need to update the entire repo. For some repo’s, this is not easy to do. Then one I’m working with currently is at least over 12 gigs, with hundreds of thousands of files, both ASCII and binary. That means that if at least 12 developers are working on it at the same time, chances are high a lot of files will be added and changed. Regardless, you only take the hit in the beginning, if you update frequently, and have a good diff tool, this increases your ability to ensure your code won’t break the build when you check in. Enterprise tools installed on the source control server itself like Cruise Control can actually automate this process, and send emails with information on who checked in code, if it compiled or not, and what was changed. To be fair, this isn’t a problem with a lot of independent server-side code.

Another minor point is how some source control systems work with locking files. Some allow developers to check out code and "lock" it. This prevents others from checking it in. Some make it impossible to work with because the code then becomes read-only. Since having another directory under your control, away from the source folder, you won’t have this problem.

To summarize, there are a lot of problems, even with built-in IDE tools, to working directly in the repo:

  1. conflict resolution prevents you from compiling. Manager says, "Can I see the latest?" Dev replies, "No… I’m merging code."
  2. folder change conflict resolution. "Can you re-factor later?"
  3. pain avoidance code committing. "I’d better hurry up and check in what I have working so I don’t have to merge… to heck with focusing".
  4. not checking out the entire repo, committing code, and breaking the build. "Works on my box…"

Two Directory Source Control Work Flow

To reiterate, Two Directory Source Control is a work flow where you utilize one directory from source control, one directory that you work out of, and you then shuttle & merge between them at your convenience. I’ve been using this work flow for about 2 1/2 years now and everyone I encounter usually finds it extremely odd I don’t work directly in the repo, hence the need for this blog entry. I was taught this work flow by Thomas Burleson, one of people responsible for the Universal Mind Cairngorm Extensions . Beyond the above, his additional justification at the time was that with Flex 1.5, SVN and Tomcat or JRun would do weird things together every so often, resulting in strange compilation bugs. Before Flex 2, you had to compile on the server (it was the law), so you’d setup a local web server on your box, and refresh a web browser page to see your app.

This work flow solves the problems above.

The first thing is conflict resolution is now done at your convenience. This is a big deal. Suddenly getting the latest from the repo is not a grand affair. You can just get the latest, and go back to work. You can also see what has changed without having to immediately integrate it into your code base if you’re not ready to do so. You can also keep working; since the code you updated is in a totally different directory, the code you’re working on isn’t affected. About to catch a plane an losing wireless? Just update, and you can still focus on coding instead of resolving your changes with someone else when you are incapable of giving them a phone call to discuss. Finally, you can confidence that when you update, your code will still work. That’s a good feeling.

Secondly, file conflict resolution is challenging, yet fun. Folder conflict resolution, however, is very challenging, and requires a lot of meticulous concentration. Deleting directories can delete a lot of files and folders within them, and thus tick a lot of other developers off if you screw up and check in your screw up. So you have to be careful. Rather than updating some massive package re-factoring and preparing for a battle with Subclipse, you can instead just let Tortoise SVN do it’s job; you just right click and update; done. When you’re ready, and you know your version of code works, you can then merge the 2 pieces using Beyond Compare, or whatever Merging tool you use. It’s a lot easier to do this when you have 2 real, physical folder directories vs. a virtual one.

Third, this convenience extends to code committing. Rather than being motivated to commit code out of fear of merging, you can do so when you’re good and ready. Even if you wait 3 days, you know you’ll have plenty of time to merge the changes into your code, get it working, THEN check in inside a safe, non-repo area. This confidence in your ability to merge with the assurance you have a nice sequestered area away from your repo motivates you to focus on creating good code vs. creating compiling code for the mere sake of making merging easier.

Fourth, because this work flow forces you to continually get the latest code form the entire repo, you always have the latest build, not just a portion of the build. This ensures that when you check in code, you know it won’t break the build because you already have the entire build and have tested it on your local box. Typically anything wrong from there is a configuration issue, not a code out of sync issue.

You need 2 things to utilize this work flow. The first is your folder structure and the second is a good merge / diffing tool.

Folder Structure

When setting up a project, you create your project folder. Inside of that, you make 2 directories called "dev" and "local". The "dev" folder is where your repo goes. This can be an SVN/CVS repo, or a Perforce work space. In the case of Subversion, I’ll usually have Tortoise check out trunk, tags, and branch here.

To say it another way, "dev" is where your source control code goes. The "local" folder is where your copy goes. You play and work inside of local. When you are ready to merge in changes, you use a merge tool to do so.

dev and local folders

Setting Up Beyond Compare

For the merge tool, I use Beyond Compare . While it’s only for PC, I purchased CrossOver , a Windows emulation tool for Mac, just so I could retain the same work flow on both my PC and Mac.

BeyondCompare has a few setup steps that you only have to do once. These are:

  1. Change default keyboard short cuts.
  2. Creating a new "session" to have your repo on the left, local code on the right.
  3. Showing only mismatches.
  4. Making your session expand folders & expand only folders with differences.
  5. Setting up BeyondCompare to filter (aka not show) certain files and folders
  6. Adjusting the file comparison control to be Size & CRC based
  7. Full Refresh!
  8. Synchronize to Right.

Let’s go over these steps in detail.

1. Change Default Keyboard Shortcuts

When you get the latest code, you’ll want to shuttle over to your local repository when you’re ready. If it’s new or unchanged files, you’ll want to do this quickly. I find Control + Right Arrow and Control + Left Arrow for putting local code todev is the best shortcuts for this.

Go to Tools > Options, and select "Keyboard" from the bottom left. Scroll down till you see "Copy to Right". Click it and press Control + Right Arrow key. Click "Copy to Left" and press Control + Left Arrow key.

BeyondCompare keyboard shortcuts

2. Creating a new "session" to have your repo on the left, local code on the right.

The easiest way to create a new session in BeyondCompare is to go "Session > New Empty". A session in BC is your saved settings with a name. A session will store what directory you are viewing on the left, what directory you are comparing to on the right, and other comparison options. You typically create 1 session per project. I typically put the dev, your source code repository on the left, and local, your copy, on the right.

Dev on the left, Local on the right

3. Showing only Mismatches

Click the "Only Mismatches" button. This will filter the files to only show files and folders that aren’t the same. Blue ones will be files that do not exist on the other side. Red indicates files that are different and gray indicates a filethat’s older than the file on the other side.

Show only mismatches

4. Making your session expand folders & expand only folders with differences

You want to have the views automatically expand folders if they are different. To do this, you need to check some check boxes in 2 places. The first place is in the Session settings under "Session > Session Manager". For your session, you’ll see a section called "Folder Handling". Check all 3 check boxes.

The second place is in "Session > Comparison Control". You’ll see the same 3 check boxes at the bottom of the dialogue; click all 3.

Expand folders automatically and only with differences

You’ll have to do this again in another dialogue; see step #6.

5. Setting up Beyond Compare to filter (aka not show) certain files and folders

There are some files you don’t care about. For Subversion, for example, these are ".svn". For Flex Builder, this includes .actionScriptProperties, .settings, etc. For Mac users who show all files, this is .DS_Store. You can filter these files out by clicking the glasses icon, and typing in the files and folders you want to filter out, 1 per line, and clicking OK.

While this will filter out files in the view, it WILL copy them if you shuttle folders back and forth. I never copy folders across inBeyond Compare , only files. The exception to this rule is if you do an "Actions > Synchronize Folders". This will obey your filter settings. If weird things start happening in your code, make sure you didn’t accidentally copy a folder with a bunch of .svn folders and files in it.

6. Adjusting the file comparison control to be Size & CRC based

To ensure you get a decent difference between files, choose both size & crc checking for file comparisons. You can choose this radio button in "Session > Comparison Control". See above picture.

modify file comparison and re-do folder expansion

7. Do a full refresh!

This will compare your entire folder & file structure. You can do so via "View > Full Refresh".

8. Synchronize to Right

If this is your first time shuttling files from your dev folder to your local folder, you can go "Actions > Synchronize Folders > Synchronize to Right".

Conclusions

That’s it, you’re ready to go! Get latest revision, do a full refresh, merge, code. Later, you can get latest revision, merge your code left, and check-in with confidence! In the end, Two Directory Source Control allows you to still compile your code when checking out the latest revision, makes it easier to resolve largere-factoring efforts, allows code committing to be a pleasurable endeavor, and ensuring you always check out the whole repo so you can confirm it builds on your machine with your latest changes. I hope it helps you out too.

Let me digress here a minute and point out how disappointed I am in the Mac platforms support of development tools. Tortoise SVN and Beyond Compare are fantastic tools for the PC. I have not found the equivalent for Mac. SvnX and the Tortoise equivalent for Mac really really suck, especially when you grew up in Tortoise SVN. Same thing on the Merge front; I have yet to find a good GUI based Merge tool for Mac. I’ll probably install Parallels on my new Mac, and stick to the PC version of Tortoise SVN & Beyond Compare since I still can’t shake Flash Develop .

0
Your rating: None