December 19, 2013

Debugging python with vim in a virtualenv

Yay I finally got it working my debugging of a python script with Vdebug in vim
Vdebug is a vim plugin that lets you debug all kinds of scripts/languages in vim.
It supports these languages:
Python
Perl
PHP
Ruby
Node.js

At work I use it to debug Perl code and I wanted to use it to debug my python code for personal projects.

the key is: pip install  komodo-python-dbgp within your virtualenv once that is done it will create the executable pydbgp in your virtualenv. so you can launch in vim vdbug and then in the shell:
pydbgp <script>

December 16, 2013

Working on a new web project

Ok, so I started working on a web project for a friend of mine.
I want to work with some new stuff so I decided to implement this application with some neat new technologies (well new for me anyway).

All the while I have been dabbling with some stuff during the years now and I came up with a basic installation to get my workflow ready.
For this I am making use of a script that I execute to setup my projects environment:


This installs an isolated python environment and an isolated node.js environment for webdevelopment.
The script will also offer the option to work either with a Yeoman workflow or a Brunch based workflow.


When I start my work I just use another script that will launch my work environment in correct settings.
For that I make use of tmux a screen multiplexer that enters in the activated virtualenv:


I've decided to build this all with Angularjs in the frontend and flask or django on the backend. The backend I will decide later on.
For the workflow I will use Brunch because it is a fairly basic webapplication and the speed of usage in a Brunch workflow outweighs the flexibility/complexity of the Yeomand/Grunt  configuration.
I made a skeleton to work with coffeescript and Angularjs and made that available on github.

I will provide details of my progress.

October 14, 2011

VLC and AirportExpress

At work we got our hands on an Airport Express. So the first thing of course is "MUSIC".
Well we do have some people here that want to use iTunes. But for personal reasons I don't like iTunes.
So I normally use VLC like any sane person would do :) But while all of my co-workers were laughing at me cuz i couldn't join with the music streaming.
Not taken aback I was strolling the internet to find if there was a solution of streaming to Airport. There was a program called Airfoil but hey I'm dutch so i really don't want to pay for programs unless necessary. At the VLC forums i stumbled on a post by crzyhanko and he posted some great code you can put in the standard streamchain field of the VLC player:
#transcode{acodec=alac,channels=2,samplerate=44100}:raop{host=<ip address of airport express>,volume=175}
It works :D so who is laughing now

March 10, 2011

SQL remove of constraints

Note to self:

when doing large imports using a sql script in oracle. here's how to remove constrains and then enable them after insert:


This code is useful to disable the constraints in the database.
set serveroutput on;
begin
  for c in (select constraint_name, table_name from user_constraints where constraint_type='R') loop
    execute immediate('alter table '||c.table_name||' disable constraint '||c.constraint_name);
  end loop;
end;
/
the '/' at the end lets sql developer know that this is the end of an inline pl/sql script

then insert the normal sql insert script and when done include this code:
begin
  for c in (select constraint_name, table_name from user_constraints where constraint_type='R') loop
    execute immediate('alter table '||c.table_name||' enable constraint '||c.constraint_name);
  end loop;
end;
/
-- SHOW ENABLED --
select constraint_name, status from user_constraints where constraint_type='R';
When the last line still shows disabled constraints the data is corrupt.

Blobs of type String can be inserted via a workaround:
declare myBLobVar varchar2(32767) := 'paste string here' ;
begin
  update tableWithBlob set blobCol = myBlobVar where id = blah ;
end;

July 20, 2009

Eclipse Templates

Templates are a usefull thing when working with code as we know.
A simple template is a simple thing to do but using an import is a different beast.

so here is a example to make sure that the import is also included in the java file.

/** Tapestry render phase method. Called before component body is rendered.*/
@BeforeRenderBody
public void beforeRenderBody(){
${cursor}
}
${:import(org.apache.tapestry5.annotations.BeforeRenderBody)}


May 21, 2009

JTA

Most applications using ORM tooling have need of a transaction management system.
One of these transaction managers is Atomikos. Atomikos provides several products. One transaction essentials is an opensource variant.
However you cannot get it via a maven repository. You'll have to register for a download link.
Transaction essentials is easy embeddable within a jetty container or even within a spring context.

First here's how to implement transaction essentials within a jetty container configured with maven.

Lets start with the jetty configuration within a maven pom file:

<!-- Run the application using "mvn jetty:run" -->
            <plugin>
                <groupId>org.mortbay.jetty</groupId>
                <artifactId>maven-jetty-plugin</artifactId>
                <version>6.1.15</version>
                <configuration>
                    <!-- Log to the console. -->
                    <requestLog implementation="org.mortbay.jetty.NCSARequestLog">
                        <!--
                        This doesn't do anything for Jetty, but is a workaround for a
                        Maven bug that prevents the requestLog from being set.
                        -->
                        <append>true</append>
                    </requestLog>
                   
                    <scanIntervalSeconds>0</scanIntervalSeconds>
                    <webAppConfig>
                      <contextPath>/rbudisplay</contextPath>
                      <tempDirectory>${project.build.directory}/work</tempDirectory>
                    </webAppConfig>
                    <jettyConfig>src/etc/jetty/tx-jetty.xml</jettyConfig>
                    <jettyEnvXml>src/etc/jetty/jetty-env.xml</jettyEnvXml>
                </configuration>
                <dependencies>
                    <dependency>
                        <groupId>javax.transaction</groupId>
                        <artifactId>jta</artifactId>
                        <version>1.1</version>
                    </dependency>
                    <dependency>
                        <groupId>com.atomikos</groupId>
                        <artifactId>atomikos-util</artifactId>
                        <version>3.5.4</version>
                    </dependency>
                    <dependency>
                        <groupId>com.atomikos</groupId>
                        <artifactId>transactions</artifactId>
                        <version>3.5.4</version>
                    </dependency>
                    <dependency>
                        <groupId>com.atomikos</groupId>
                        <artifactId>transactions-api</artifactId>
                        <version>3.5.4</version>
                    </dependency>
                    <dependency>
                        <groupId>com.atomikos</groupId>
                        <artifactId>transactions-jta</artifactId>
                        <version>3.5.4</version>
                    </dependency>
                    <dependency>
                        <groupId>com.atomikos</groupId>
                        <artifactId>transactions-hibernate3</artifactId>
                        <version>3.5.4</version>
                    </dependency>
                    <dependency>
                        <groupId>postgresql</groupId>
                        <artifactId>postgresql</artifactId>
                        <version>8.3-603.jdbc4</version>
                    </dependency>
                    <dependency>
                        <groupId>log4j</groupId>
                        <artifactId>log4j</artifactId>
                        <version>1.2.14</version>
                    </dependency>
                </dependencies>
            </plugin>

The jettyConfig tx-jetty contains the configuration for the atomikos usertransactionmanager and will be applied before other setting set in the maven pom:

<Call class="java.lang.System" name="setProperty">
      <Arg>com.atomikos.icatch.file</Arg>
      <Arg>src/etc/jetty/jta.properties</Arg>
   </Call>

   <!-- Atomikos -->
   <New id="tx" class="org.mortbay.jetty.plus.naming.Transaction">
      <Arg>
         <New class="com.atomikos.icatch.jta.UserTransactionImp" />
      </Arg>
   </New>

The jetty-env contains the configuration for a specif webapplication in jetty and consists in part of the datasources for the webapp:

<Set name="configurationClasses">
    <Array type="java.lang.String">
      <Item>org.mortbay.jetty.webapp.WebInfConfiguration</Item>
      <Item>org.mortbay.jetty.plus.webapp.EnvConfiguration</Item>
      <Item>org.mortbay.jetty.annotations.Configuration</Item>
      <Item>org.mortbay.jetty.webapp.JettyWebXmlConfiguration</Item>
      <Item>org.mortbay.jetty.webapp.TagLibConfiguration</Item>
    </Array>
  </Set>
 
  <!-- Add a mapping from name in web.xml to the environment -->
  <New id="map1" class="org.mortbay.jetty.plus.naming.Link">
    <Arg><Ref id='rbudisplay'/></Arg>
    <Arg>jdbc/rbuconverter</Arg> <!-- name in web.xml -->
    <Arg>jdbc/rbu</Arg>  <!-- name in environment -->
  </New>
 
  <New id="rbuconverter" class="org.mortbay.jetty.plus.naming.Resource">
     <Arg><Ref id="rbudisplay"/></Arg>
     <Arg>jdbc/rbu</Arg>
     <Arg>
      <New class="com.atomikos.jdbc.AtomikosDataSourceBean">
       <Set name="minPoolSize">2</Set>
       <Set name="maxPoolSize">50</Set>
       <Set name="xaDataSourceClassName">org.postgresql.xa.PGXADataSource</Set>
       <Set name="UniqueResourceName">rbuconverter</Set>
       <Get name="xaProperties">
         <Call name="setProperty">
            <Arg>databaseName</Arg>
            <Arg>rbuconverter</Arg>
         </Call>
         <Call name="setProperty">
           <Arg>serverName</Arg>
           <Arg>localhost</Arg>
         </Call>
         <Call name="setProperty">
            <Arg>portNumber</Arg>
            <Arg>5432</Arg>
         </Call>
         <Call name="setProperty">
           <Arg>user</Arg>
           <Arg>postgres</Arg>
         </Call>
         <Call name="setProperty">
           <Arg>password</Arg>
           <Arg>BlahBlah</Arg>
         </Call>
       </Get>
      </New>
     </Arg>
   </New>

The first part sets up the jetty plus environment for jndi.
The second part sets up the reference for the jetty-env configuration and to bind the datasource to jndi so the web.xml can make a refernce to the configured datasource.
The third part sets up the datasource itself. Here an XADataSource is configured for PostgreSQL Note that the com.atomikos.jdbc.AtomikosDataSourceBean is the prefered DataSource since Atomikos 3.4.x.

The web.xml needs to have these lines in place for the above configured datasource:

<resource-ref>
      <res-ref-name>jdbc/rbuconverter</res-ref-name>
      <res-type>javax.sql.DataSource</res-type>
      <res-auth>Container</res-auth>
   </resource-ref>

It is also possible to configure Atomikos transaction Essentials completely in a spring context. In this case do not use the jettyConfig and jettyEnvXml in the pom file and omit the resource-ref within the web.xml:
I have a tx-context.xml:

<bean class="com.atomikos.jdbc.AtomikosDataSourceBean" destroy-method="close" id="dataSource" init-method="init">
        <property name="uniqueResourceName" value="rbudatasource"/>
        <property name="xaDataSourceClassName" value="org.postgresql.xa.PGXADataSource"/>
        <property name="xaProperties">
              <props>
              <prop key="databaseName">rbuconverter</prop>
              <prop key="serverName">localhost</prop>
              <prop key="portNumber">5432</prop>
              <prop key="user">postgres</prop>
              <prop key="password">BlahBlah</prop>
              </props>
        </property>
        <property name="minPoolSize" value="5">
        <property name="maxPoolSize" value="50">
    </bean>

    <bean class="com.atomikos.icatch.config.UserTransactionServiceImp" destroy-method="shutdownForce" id="userTransactionService">
        <constructor-arg>
             <props>
                <prop key="com.atomikos.icatch.service">
                    com.atomikos.icatch.standalone.UserTransactionServiceFactory
                </prop>
                <prop key="com.atomikos.icatch.output_dir">/tmp</prop>
                <prop key="com.atomikos.icatch.output_dir">/tmp</prop>
            </props>
        </constructor-arg>
   </bean>           
 
   <!-- Construct Atomikos UserTransactionManager, needed to configure Spring -->
   <bean id="AtomikosTransactionManager" class="com.atomikos.icatch.jta.UserTransactionManager" init-method="init" destroy-method="close" depends-on="userTransactionService">
      <!-- when close is called, should we force transactions to terminate or not? -->
      <property name="forceShutdown" value="false" />
      <property name="transactionTimeout" value="300"/>
   </bean>

   <!-- Also use Atomikos UserTransactionImp, needed to configure Spring -->
   <bean id="AtomikosUserTransaction" class="com.atomikos.icatch.jta.UserTransactionImp" />

   <!-- Configure the Spring framework to use JTA transactions from Atomikos -->
   <bean id="transactionManager" class="org.springframework.transaction.jta.JtaTransactionManager" depends-on="userTransactionService">
      <property name="transactionManager" ref="AtomikosTransactionManager" />
      <property name="userTransaction" ref="AtomikosUserTransaction" />
   </bean>

April 4, 2009

Tapestry activation passivation

Tapestry 5 can make use of the client to make information set to a page from another page available to the client.
For instance Page Origin places a message during a submit onto Page Next and using the mechanism of passivation and activation
the client will act as the middle man to make this information available to the client itself.
Tapestry will issue an HTTP 302 redirect after the post and thus the passivation of Page Next so the client recieves some information and will ask tapestry for that information to be acted upon.
Here you see that the value of 'Howdy' is submitted and using a redirect is send to the server again to be used in its activation event.

February 2, 2009

Apache2 VirtualHost and trac behind a lighttpd proxy

Note to self:
setup of trac for Apache2

follow this instructions on this site: Robert Basic: Trac on Ubuntu.
Although I use mercurial as scm.
make sure that the line containing VirtualHost points to port 81 or another port except port 80.

<VirtualHost *:81>
and make sure apache listens to this port by changing /etc/apache2/ports.conf
NameVirtualHost *:81
Listen 81

Now install lighttpd:

sudo apt-get install lighttpd

edit /etc/lighttpd/conf-available/10-proxy

$HTTP["host"] == "localhost" {
proxy.debug = 1
proxy.server     = ( "/trac" =>
                     ( 
                       ( "host" => "127.0.0.1",
                         "port" => 81,
                       )
                     )
                    )
}
and symlink:
sudo ln -s /etc/lighttpd/conf-available/10-proxy /etc/lighttpd/conf-enabled/10-proxy


and start lighttpd
sudo /etc/init.d/lighttpd restart
et voilĂ 
the trac instance works both under localhost/trac as well as localhost:81/trac

January 5, 2009

Setting up my python development environment

Note to self:
my setup for a python development environment
sketch so far:


use virtualenv wrapper changed mkvirtualenv function to to virtualenvwrapper_bashrc
function mkvirtualenv () {
    verify_workon_home
    (cd "$WORKON_HOME"; virtualenv $*)
    if [ ! -f $WORKON_HOME/$1/bin/postactivate ]
    then
        workon "${@:-1}"
        easy_install ipython
        easy_install pysmell
        wget http://www.eletztrick.de/software/mkvimproject/releases/0.4.2.3/mkvimproject-0.4.2.3.tar.gz
        easy_install mkvimproject-0.4.2.3.tar.gz
        rm mkvimproject-0.4.2.3.tar.gz
        deactivate
        touch $WORKON_HOME/$1/bin/postactivate
        postactivate="$WORKON_HOME/$1/bin/postactivate"
        echo "cd $WORKON_HOME/$1" >> $postactivate 
        echo "export PYTHONPATH=$WORKON_HOME/$1/scr:$PYTHONPATH" >> $postactivate    
        echo "if [ ! -d $WORKON_HOME/$1/src ]" >> $postactivate
        echo "then" >> $postactivate
        echo "    mkdir -p $WORKON_HOME/$1/src" >> $postactivate
        echo "fi" >> $postactivate
        echo "if [ ! -f $WORKON_HOME/$1/$1.vpj ]" >> $postactivate
        echo "then" >> $postactivate
        echo "    mkvimproject -o $1.vpj -s python" >> $postactivate
        echo "fi" >> $postactivate
        echo "if [ ! -f $WORKON_HOME/$1/PYSMELLTAGS ]" >> $postactivate
        echo "then" >> $postactivate
        echo "    pysmell src;" >> $postactivate
        echo "  cd lib/python2.5;" >> $postactivate;
        echo "  pysmell . -x site-packages -o ../../PYSMELLTAGS.stdlib;" >> $postactivate
        echo "  cd ../..;" >> $postactivate 
        echo "fi" >> $postactivate
        echo "pproject -U" >> $postactivate
    fi
    workon "${@:-1}"
}

then create a new virtual env with mkvirtualenv projectName --no-site-packages
this will setup the environment for vim and allow for the use of pysmell omnicompletion
and allows a ipython shell within the environment.

April 8, 2008

Google app engine

On april 7 google released the google app engine.
This is great news.

An application environment that can host with the scalability of google itself.
And whats even better currently the engine is implemented in python.

Python is a dynamic language a la ruby. And created by a Dutchman ;-)
It's fully object-oriented. and it is fun to work with.
here is a video on how to develop with the google app engine



As you can see the development process is way faster than the development process of lets say Java.

Here is a video of the introduction of app:



I already have an account and will experiment with in the near future.
BTW This is a real cool way to start working with Django