Split my local SVN archive

Among the years spent at IMT, my SVN repository has become bloated with many things, some of which are not directly related to the work done at Parlab. Before the group closes, I plan to release the software I wrote as open-source, and hopefully with the full SVN history of this software. This requires to split up my bloated repository into smaller units.

I tried to go the 'svnadmin dump' 'svndumpfilter' way, but this did not work due to my abuse usage of cheap copies (thank to the TortoiseSVN copy option). The 'svndumpfilter2' and 'svndumpfilter3' replacements did not work either.

svndumpfilter failed on the first revision with a copy from a non-included path

svndumpfilter2 produced a dumpfile, but displayed various errors from svnlook while running.

The dumpfile could not be exported to a repository.

svndumpilter3 seemd to process everything, but produced many strange beeps during exectution, probably caused by invalid UTF-8 interpretations of binary data (png images and so on). And I failed to find the output dumpfile :-/ Maybe clearer usage explenations from the author would have helped.

Right now, I'm trying to follow the instructions by Cott Lang at Internet Stall, who seems to have encountered the same issues I had with 'svndumpfilter*'

http://www.internetstaff.com/roller/blog/date/20090625

The target repository for sync is created by :

jamesMBpro:tempSVN james$ svnadmin create --fs-type fsfs /tempSVN/FSFS/docREP/

and now, the svnsync preparation, according to http://svnbook.red-bean.com/en/1.5/svn.reposadmin.maint.html#svn.reposadmin.maint.replication

First, change the pre-revprop-change script (done in step 3 in Cott Lang's howto):

jamesMBpro:tempSVN james$ smultron ./FSFS/docREP/hooks/pre-revprop-change

#!/bin/sh   USER="$3"  if [ "$USER" = "syncuser" ]; then exit 0; fi  echo "Only the syncuser user may change revision properties" >&2 exit 1

and then, change the start-commit script:

jamesMBpro:tempSVN james$ smultron ./FSFS/docREP/hooks/start-commit

#!/bin/sh   USER="$2"  if [ "$USER" = "syncuser" ]; then exit 0; fi  echo "Only the syncuser user may commit new revisions" >&2 exit 1

svnsync initialization:

jamesMBpro:tempSVN james$ svnsync initialize file:///tempSVN/FSFS/docREP file:///tempSVN/svnIMTtemp/trunk/doc --sync-username syncuser --sync-password syncpwd

Copied properties for revision 0.

jamesMBpro:tempSVN james$

then, the copying in itself:

jamesMBpro:tempSVN james$ svnsync synchronize file:///tempSVN/FSFS/docREP --sync-username syncuser --sync-password syncpwd

Committed revision 1.

Copied properties for revision 1.

Committed revision 2.

Copied properties for revision 2.

Committed revision 3.

Copied properties for revision 3.

.

.

.

Transmitting file data ...........

Committed revision 2597.

Copied properties for revision 2597.

Transmitting file data .

Committed revision 2598.

Copied properties for revision 2598.

Transmitting file data ...

Committed revision 2599.

Copied properties for revision 2599.

jamesMBpro:tempSVN james$

Now, check the copied repository.

jamesMBpro:~ james$ cd /tempSVN/coTest/

jamesMBpro:coTest james$ mkdir docTest

jamesMBpro:coTest james$ cd docTest/

jamesMBpro:docTest james$ svn checkout file:///tempSVN/FSFS/docREP

A docREP/trunk

A docREP/trunk/doc

.

.

.

A docREP/trunk/doc/JMuWebsite/WebContent/default.php

A docREP/trunk/doc/JMuWebsite/WebContent/video_occlusion.php

Checked out revision 2599.

jamesMBpro:docREP james$

One potential hickup here. The synced repository has the same directory structure as the original repository had (in this case, trailing /trunk/doc/). Nevertheless, the checkout seemed to work, let's see what the log has to say.

jamesMBpro:docTest james$ cd docREP/

jamesMBpro:docREP james$ svn log

------------------------------------------------------------------------

r2599 | james | 2009-10-19 19:07:33 +0200 (Mon, 19 Oct 2009) | 2 lines

+ changed website links to include thesis draft, presentations

+ fixed some of www-imt dead links

------------------------------------------------------------------------

r2598 | muredubo | 2009-10-16 16:00:25 +0200 (Fri, 16 Oct 2009) | 1 line

done ?

------------------------------------------------------------------------

r2597 | muredubo | 2009-10-16 15:36:45 +0200 (Fri, 16 Oct 2009) | 1 line

almost done

.

.

.

------------------------------------------------------------------------

r4 | mtzamofing | 2004-11-26 08:39:13 +0100 (Fri, 26 Nov 2004) | 2 lines

*** empty log message ***

------------------------------------------------------------------------

r3 | mtzamofing | 2004-11-25 14:59:38 +0100 (Thu, 25 Nov 2004) | 2 lines

*** empty log message ***

------------------------------------------------------------------------

r2 | mtzamofing | 2004-11-25 14:38:34 +0100 (Thu, 25 Nov 2004) | 2 lines

*** empty log message ***

------------------------------------------------------------------------

r1 | (no author) | 2004-11-25 14:38:34 +0100 (Thu, 25 Nov 2004) | 1 line

Standard project directories initialized by cvs2svn.

------------------------------------------------------------------------

jamesMBpro:docREP james$

Building of one of the most important LaTeX document in this repository (my PhD thesis) worked flawlessly on the first try. Of course, more validation of the robustness of this copy is required, but this is encouraging.

Updates, Quicktime, Nvidia Quadro and BSODs

I regularly update the software on the Quad Core machine I use for development in our lab: a Dell Precision 390 with a Nvidia Quadro FX 3450 graphics card. Recently, I began experiencing problems which seemed related to the graphics card. The display would freeze indefinitely, or change to the infamous BSOD.



Sometimes upon starting or restarting the machine, the display would be garbled, as if the VGA connector wasn't properly connected (it was). On various occasions, Windows detected a corrupt disk upon restart, but the automated chkdsk seemed to always be able to fix the problem (in 30 minutes).

Remembering that the latest Windows update included an update to the Nvidia driver, I thought I had found the culprit. I foolishly tried to roll back to Dell's driver, from Dell's website.

http://support.euro.dell.com/support/downloads/driverslist.aspx?c=ch&l=en&s=gen&ServiceTag=&SystemID=LATITUDE%20PRECISION

%20390&os=WW1&osl=en&catid=&impid=

Note that apparently Dell also provides a vintage Quadro FX3450 BIOS (v5.41.02.51.01 from 01.12.2006) ! The old driver (v84.26 from 01.11.2007) seemed to install correctly, until it asked permission to restart the machine. Upon login after the restart, the driver installation program resumed, and managed to garble the display in ways that appear quite artistic in retrospective.

One hard reboot later, I decided to let Dell's abandon-driver to rest, and proceeded to download and install NVidia newest driver.

http://www.nvidia.com/object/Quadro_winxp_x32_191.00_whql.html

Alas, the bluescreen problem reappeared after only a few hours. Seriously bedazzled, I did waht I should have done first, I fired MS computer management utility, to look at error logs.

And there it was, numerous warnings about paging errors, followed by critical errors attributed either to my system drive, or to iaStor.sys This got me worried about a possible failure of the hard disk drive.

I remembered that while I was updating the system through Windows update, I had selected to install the optional Intel Matrix Storage software. But had finally not activated it since the system contains only 2 hard drives, on used for WinXP, and the other one for CentOS.

To my relief, hardware failure became less likely after a quick google search turned out many disturbing links, suggesting that Dell and iaStor are a risky combination :

http://www.google.com/search?rls=en&q=dell+precision+390+nvidia+iaStor&ie=UTF-8&oe=UTF-8

see in particular:

Things Worth Knowing » Blog Archive » Dell Dimension E510, iastor ...

Shortly after, I stumbled on the following link :

http://www.istartedsomething.com/20070712/quicktime-bug-nvidia-sata-vista/

And then, bing (no pun intented), it hit me. I had also been asked by Apple update if I wanted a new version of iTunes and Quicktime. I decided to pass on iTunes, but said Ok to Quicktime.

Note that the post linked above concerns Vista, and I'm running XP. But the message I got is that Quicktime can induce trouble with iaStor on Dell Precision 390 systems.

Since I did not need the RAID software, I first uninstalled Intel Matrix Storage. However, this did not prevent the BSODs from returning. Which suggests that this software is not responsible for the crashes I experienced.

So, I uninstalled Quicktime, and tada, the crashes stopped. For now, I'm a happy camper (without Quicktime), and I can go back to my spherical Fourier transforms for DynamicVA.

But I really hope Apple fixes wahtever is wrong with QuickTime.

FFTW issues in Matlab on Mac

For the project I'm currently working on (dVA on sphere), I'm using the Yawtb toolbox, with the S2Kit interface (see previous post) . This interface is faster when it uses FFTW3.

I followed the build instructions for S2Kit, using Macports's FFTW. S2Kit built and tested ok from command line. Testing the integration in Yawtb, I encountered a linker error:

>> yademo s2fst

% Sample(s) of code for S2FST

%

%

>> [phi,theta] = sphgrid(46,46); %%

>> mat = exp(-tan(theta/2).^2) .* ( phi == phi(1,10)); %%

>> fmat = s2fst(mat); %%

??? Invalid MEX-file '/IM2sandbox/yawtb/interfaces/s2kit/s2fst.mexmaci':

dlopen(/IM2sandbox/yawtb/interfaces/s2kit/s2fst.mexmaci, 1): Library not loaded: /opt/local/lib/libfftw3.3.dylib

Referenced from: /IM2sandbox/yawtb/interfaces/s2kit/s2fst.mexmaci

Reason: Incompatible library version: s2fst.mexmaci requires version 6.0.0 or later, but libfftw3.dylib provides version

5.0.0.

Error in ==> yademo at 77

eval(subcode(3:end));

Some googling after (see comments here), I learned that this comes from a version mismatch between Matlab's FFTW and the newer Macports version.

Contrary to Geoffrey, I decided to update the lib in Matlab (quite similarly to the suggestion here). Note: a backup is recommended. In my case, the fix implied copying libfftw3.dylib and libfftw3.dylib from /opt/local/lib to /Applications/MATLAB_R2008b.app/bin/maci/ .

And now, the yademo s2fst example works great :-)

Note that I did not check whether this fix breaks other FFTW related code. YMMV.

Spherical wavelets toolbox - run on windows

I recently began to work on a project concerning dynamic visual attention (VA) on the sphere, and the current code base uses Yawtb and SpharmonicKit.

Since the camera used in the project has only MS Windows drivers, I wanted to run the algorithms in the same OS. A quick google search turned only to posts with similar questions (see questions on Chris Rodgers software blog).

So, I took the plunge and tried to compile on my WinXP SP3 box, with MS Visual Studio 2005. I used premake to create the target projects, based on a quick look at the Makefiles. Note that I replaced the SpharmonicKit27 interface with the more recent S2Kit, which uses fftw for faster Fourier transforms.

Apparently, MS compilers have gained in compliance to ANSI standards over the years. The only issues I had were:

• undefined M_PI for both Yawtb and S2Kit

fixed with '#define _USE_MATH_DEFINES' in files including math.h

• srand48() and drand48() could not be found (for S2Kit test files)

here, the fix is less obvious. For now I just used rand() and (double) rand() / (double) RAND_MAX , but this is probably far from optimal (please don't flame for not using your favorite RNG ;-).

A copy of the modified S2Kit is available at

https://documents.epfl.ch/users/m/mu/muredubo/www/s2kit10.zip

while the modified Yawtb is at:

https://documents.epfl.ch/users/m/mu/muredubo/www/yawtb.zip

Beware, those quick patches run on my XP machine, but have not been thoroughly tested. Your mileage may vary.

Windows explorer crashes when external HDD is connected

I'm running XP SP3, and trying to finish my PhD thesis defense presentation. I had to connect my USB external HDD to find some videos I had archived, because I wanted to include them in the presentation.
As I took a break , the screensaver kicked in. When I came back, Windows Explorer had crashed. My machine is currently almost unresponsive.
I googled my problem, and quickly found links suggesting that this crash could be related to windows creating thumbs for the videos on the external drive.
A fix is proposed by Wile E at
http://forums.techpowerup.com/showthread.php?t=54365:

regsvr32 /u shmedia.dll

The indexing service popped a mesage box to indicate it stopped, but I'm still unable to restart explorer. So, my next step will be to pull the plug on my external HDD, and reboot. Right now, I cross my fingers hoping that the partitions will still be readable when I plug teh disk back in.

Latex - Beamer warnings after update

While preparing my PhD defense presentation, in ran into the warning:

Package hyperref Warning: Option `pdfpagelabels' is turned off
(hyperref) because \thepage is undefined.
Hyperref stopped early

Google was my friensd this day since the first search result provided the appropriate workaround:
http://www.latex-community.org/forum/viewtopic.php?f=5&t=162
\documentclass[hyperref={pdfpagelabels=false}]{beamer}

Cheers to localghost, whose answer allowed me to skip reading the change logs of the hyperref package at such a stressful time.

Edit:
In fact, all the new warnings I got were related to the update of beamer and hyperref.
This blog entry : http://texblog.net/latex-archive/presentations/beamer-warnings/ presents a step by step workaround. Thanks Stefan.

Cross compilation and string issues

Trying to keep the program I am currently working on up-to-date both on my WinXP box and on a Mac, I run into some cross-compilation pitfalls. While the project builds without warnings in vs2005, it seems that gcc on Mac has a lot to complain about.

First of all some deprecated headers apparently used by the macports VTK I'm using:

In file included from /usr/include/c++/4.0.0/backward/strstream:51,
             from
../../../../opt/local/var/macports/software/vtk-devel/5.4.0_3+boost+cocoa+darwin_9+data+doc+examples+java+py26+shared+tcl+testing+wrap/opt/local/include/vtk-5.4/vtkIOStream.h:112,
             from
../../../../opt/local/var/macports/software/vtk-devel/5.4.0_3+boost+cocoa+darwin_9+data+doc+examples+java+py26+shared+tcl+testing+wrap/opt/local/include/vtk-5.4/vtkSystemIncludes.h:40,
             from
../../../../opt/local/var/macports/software/vtk-devel/5.4.0_3+boost+cocoa+darwin_9+data+doc+examples+java+py26+shared+tcl+testing+wrap/opt/local/include/vtk-5.4/vtkIndent.h:24,
             from
../../../../opt/local/var/macports/software/vtk-devel/5.4.0_3+boost+cocoa+darwin_9+data+doc+examples+java+py26+shared+tcl+testing+wrap/opt/local/include/vtk-5.4/vtkObjectBase.h:43,
             from
../../../../opt/local/var/macports/software/vtk-devel/5.4.0_3+boost+cocoa+darwin_9+data+doc+examples+java+py26+shared+tcl+testing+wrap/opt/local/include/vtk-5.4/vtkObject.h:41,
             from
../../../../opt/local/var/macports/software/vtk-devel/5.4.0_3+boost+cocoa+darwin_9+data+doc+examples+java+py26+shared+tcl+testing+wrap/opt/local/include/vtk-5.4/vtkAlgorithm.h:32,
             from
../../../../opt/local/var/macports/software/vtk-devel/5.4.0_3+boost+cocoa+darwin_9+data+doc+examples+java+py26+shared+tcl+testing+wrap/opt/local/include/vtk-5.4/vtkPolyDataAlgorithm.h:37,
             from
../../../../opt/local/var/macports/software/vtk-devel/5.4.0_3+boost+cocoa+darwin_9+data+doc+examples+java+py26+shared+tcl+testing+wrap/opt/local/include/vtk-5.4/vtkCubeSource.h:24,
             from CamVtkView.h:14,
/usr/include/c++/4.0.0/backward/backward_warning.h:32:2: warning:
#warning This file includes at least one deprecated or antiquated
header. Please consider using one of the 32 headers found in section
17.4.1.2 of the C++ standard. Examples include substituting the 
header for the  header for C++ includes, or  instead of
the deprecated header . To disable this warning use
-Wno-deprecated.

But other warnings allowed me to spot some bug of my own producing:

CamFrame.cpp:927: warning: cannot pass objects of non-POD type 'class
wxString' through '...'; call will abort at runtime

the line was :

curImg.sprintf("%s%04u.vtk", _vtkRecPrefix, m_nFrmRead);

while what I intended to do was:

curImg.Printf(wxT("%s%04u.vtk"), _vtkRecPrefix, m_nFrmRead);

And I found a link which summarizes well wxString usage:
http://wiki.wxwidgets.org/Converting_everything_to_and_from_wxString

Easy rotation with Eigen quaternions

Currently working on improving an alignment software for time-of-flight depth maps, I have to say I find the functions offered by Eigen quite useful.
Considering the two points clouds below (rendered with VTK), the first step to align them is to rotate them correctly.
















For this task, I use a simple RANSAC to find the largest plane in the point cloud. Then, having the plane normal vector nVec3, finding the rotation matrix mat that will align this plane with the Z-axis is 2 Eigen calls away :

Eigen::Vector3d nVecZ; nVecZ.UnitZ();
Eigen::Quaterniond qz; qz.setFromTwoVectors(nVec3, nVecZ);
Eigen::Matrix3d mat = qz.toRotationMatrix();

And here is the result for the dataset above:



That's 2 rotation DoF taken care of. Imposing the matched planes to have the same distance to origin takes care of one translation DoF. One rotation and 2 translations to go ;-)

wxEventHandler function do not support overloading

After some refactoring of some code I wrote to set the visibility and color of VTK objects with GUI controls from wxWidgets, I had two overloaded functions :

/* acting on visVTK checkbox*/
void MainWnd::SetVisVtk(wxCommandEvent& event)
{
SetVisVtk(_visVtk, m_camFrm,0);
SetVisVtk(_visBGVtk, m_camFrm,1);
SetVisVtk(_visFGVtk, m_camFrm,2);
SetVisVtk(_visSegmVtk, m_camFrm,3);
_vtkWin->Render();
}
/* acting on visVTK checkbox*/
void MainWnd::SetVisVtk(std::vector visChkBox, std::vector camFrm, int idx)
{
int i = 0;
std::vector::iterator itCtrl;
std::vector::iterator itCam;
for ( itCtrl = visChkBox.begin(), itCam = camFrm.begin();
itCtrl != visChkBox.end() , itCam != camFrm.end();
itCtrl++, itCam++, i++ )
{
if(!_vtkWin){return;};
(*itCam)->hideDataActVtk( !((*itCtrl)->IsChecked()) , idx);
}
}

The code compiled and ran fine on my windows box, but trying to compile on Mac revealed the error. Renaming the worker function to avoid overloading allowed a succesful compilation.

/* acting on visVTK checkbox*/
void MainWnd::SetVisVtk(wxCommandEvent& event)
{
SetVisVtkIdx(_visVtk, m_camFrm,0);
SetVisVtkIdx(_visBGVtk, m_camFrm,1);
SetVisVtkIdx(_visFGVtk, m_camFrm,2);
SetVisVtkIdx(_visSegmVtk, m_camFrm,3);
_vtkWin->Render();
}
/* acting on visVTK checkbox*/
void MainWnd::SetVisVtkIdx(std::vector visChkBox, std::vector camFrm, int idx)
{
int i = 0;
std::vector::iterator itCtrl;
std::vector::iterator itCam;
for ( itCtrl = visChkBox.begin(), itCam = camFrm.begin();
itCtrl != visChkBox.end() , itCam != camFrm.end();
itCtrl++, itCam++, i++ )
{
if(!_vtkWin){return;};
(*itCam)->hideDataActVtk( !((*itCtrl)->IsChecked()) , idx);
}
}

Of course this should have been obvious from the start, but this is why testing on multiple platform is such a pain. The faulty code compiled flawlessly in vs2005.

Mercurial on Mac - UTF-8 locale problems

The Eigen project recently switched to Mercurial. After installing the Macports version of mercurial, it wouldn't work apparently due to locale problems. The solution from this blog post worked for me :

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

Thanks Lothar. It's really good when the first hit from a google search is spot on. Now I should get back to my own code and build problems.

SVN checkout from local repository

Since my personal repository is hosted on a win server at my university, I use rsync to copy the repository locally on macs. This avoids the hassle of trying to figure out how to use a smb URL with svn.

During the process of moving to a new machine, I copied the repository on this machine, and tried to checkout :

muredubo$ svn co /Users/muredubo/rsynRep/ /Users/muredubo/svnSandBox/

svn: '/Users/muredubo/rsynRep' does not appear to be a URL

What I had just forgotten is that even a local repository must be accessed as a URL. This means using file:/// with the critical triple slash (I had searched for it long ago, and can't remenber now who I am to thank for this trick). The command:

muredubo$ svn co file:///Users/muredubo/rsynRep/ /Users/muredubo/svnSandBox/

created my sandbox without a glitch.

New install on MBP

Since I inherited an orphan MacBookPro in our lab, I decided to use it to test my cross platform builds. But this is not so straightforward. Prior to testing, I need to put some software / libraries on it.

So far :

• apple system updates
• xmarks for safari
• firefox 3.0.10
• quick locale switcher
  
• dowload statusbar
  
• flagfox
  
• xcode 3.1
• macports 1.7.1
• qt4-mac-devel from macports (took 4 hours to build (counting dependencies ^^))
• cisco AnyConnect ( EPFL VPN )
• mactex 2008
• smultron
• wxWidgets 2.8.10 from macports
• macports python25
• macports python26
• VTK from macports with boost and wrap variants (shared, examples and data variants are default)
• premake

Todo:

• Enthought Python
• code-blocks
• many more that I forget

Edit:

found a similar blog posting with nice list of apps: http://blog.hyperjeff.net/?p=148. I'll have to take a look at the python distros used here.

Opening multiple TexnicCenter instances

On windows, I like the Texniccenter LaTeX frontend. In my opinion, it is a very good tool for handling large Latex projects. One feature I really appreciate is the document structure tree which makes navigation in the LaTeX source much easier. One issue I had is that if you open a new project, it replaces the project in the open instance (at least in the version I use: 1.0 RC1).
Fortunately, there is a trick : if you open the main tex file, with same name as the project, Texniccenter loads a new instance, and then asks if it should load the project. This allows to have multiple instances of Texniccenter. I use this a lot when writing my thesis, to copy/paste the code for figures / tables that were published in articles I wrote.

First experiments with premake4 on mac

After having delayed my transition to the new version 4 of premake for quite a long time, I'm trying to switch over without breaking too many things. I did try to switch to premake4 when it was released a few months ago, but was not able to correctly set it to build the Mac dylibs I needed. Therefore I stuck with premake3.

Now, I'm trying again. First, with the XML helper library TICPP ticpp - Google Code

premake4 --ticpp-shared gmake

which seems to work well

ticpp

TiCPP

Building configurations...

Running action 'gmake'...

Generating Makefile...

Generating TiCPP.make...

Done.

jamesMBpro:ticpp james$ make

==== Building TiCPP ====

tinyxml.cpp

tinyxmlparser.cpp

Linking TiCPP

Let's see if the dylibs were produced. And ... nope, only .so :-/

And apparently the dylib flag from premake3 was not conserved in premake4.

Next stop : try the premake4.1 beta. Unfortunately, this did not behave differently.

After some googling which returned nothing, I followed the premake forums down to the buglist. It seems this is a bug which has (recently) been submitted :

http://sourceforge.net/tracker/?func=detail&aid=2790865&group_id=71616&atid=531878

And, hurray, there is a patch ! Thank you ashberlin.

So, I downloaded the svn source for premake4:

svn co https://premake.svn.sourceforge.net/svnroot/premake/trunk premake4

And then applied ashberlin's patch

patch -p 1 < 0001-SharedLibs-on-macosx-should-be-called-.dylib.patch

And finally the dylib built. I'll check them tomorrow.