Code Answer: 05/06/11

Friday, May 6, 2011

Write php array in HBase using thrift

I have a Thrift php client and I want to write in a HBase table and I'm doing the following:

  $mutations = array(
    new Mutation( array(
      'column' => 'entry:num',
      'value' => array('a','b','c')
    ) ),
  );
  $client->mutateRow( $t, $row, $mutations );

The problem is that when inserting in HBase the value, which is an array, gets converted to 'Array' instead of storing the elements of the array. How can I store the list as an array (or byte array)

From stackoverflow

I must admit that I do not have a clue what you're trying to do (perhaps due to a lack of knowledge regarding Thrift and HBase), but if I understood your question correctly, you're trying to write some PHP data structure (array in this case) to a storage media. To achieve this you have to serialize your data somehow. That could be using a custom XML serialization, a custom binary serialization or, perhaps the most simple solution, the PHP internal serialization mechanism provided by serialize() and the corresponding unserialize().

If you strive for inter-language-interoperability you should use a custom serialization or you have to write a unserialization function that unserializes the PHP serialization format in your target language.

Just a quick example - I don't know where you'd have to put this code, as I don't know exactly what you're doing:
```
$mutations = array(
    new Mutation(array(
      'column' => 'entry:num',
      'value'  => array('a','b','c')
    )),
  );
$data = serialize($mutations); // $data now is a string
// write $data to storage
// read $readData from storage
$readMutations = unserialize($readData);
// $readMutations == $mutations 
// (but the Mutation instances are not the same instances any more)
```
Please seee

How can I read SMS messages from the inbox programmatically in Android?

I want to retrieve the SMS messages from the inbox and display them?

From stackoverflow

It is a trivial process. You can see a good example in the source code SMSPopup

Examine the following methods:

public static SmsMmsMessage getSmsDetails(Context context,
                        long ignoreThreadId, boolean unreadOnly)
public static long findMessageId(Context context, long threadId, long _timestamp, int messageType
public static void setMessageRead(Context context, long messageId, int messageType)
public static void deleteMessage(Context context, long messageId, long threadId, int messageType)

this is the method for reading:

        public static SmsMmsMessage getSmsDetails(Context context,
                        long ignoreThreadId, boolean unreadOnly) {

                String SMS_READ_COLUMN = "read";
                String WHERE_CONDITION = unreadOnly ? SMS_READ_COLUMN + " = 0" : null;
                String SORT_ORDER = "date DESC";
                int count = 0;

                //Log.v(WHERE_CONDITION);

                if (ignoreThreadId > 0) {
//                      Log.v("Ignoring sms threadId = " + ignoreThreadId);
                        WHERE_CONDITION += " AND thread_id != " + ignoreThreadId;
                }

                Cursor cursor = context.getContentResolver().query(
                                SMS_INBOX_CONTENT_URI,
                      new String[] { "_id", "thread_id", "address", "person", "date", "body" },
                                WHERE_CONDITION,
                                null,
                                SORT_ORDER);

                if (cursor != null) {
                        try {
                                count = cursor.getCount();
                                if (count > 0) {
                                        cursor.moveToFirst();

//                                      String[] columns = cursor.getColumnNames();
//                                      for (int i=0; i<columns.length; i++) {
//                                              Log.v("columns " + i + ": " + columns[i] + ": "
//                                                              + cursor.getString(i));
//                                      }

                                        long messageId = cursor.getLong(0);
                                        long threadId = cursor.getLong(1);
                                        String address = cursor.getString(2);
                                        long contactId = cursor.getLong(3);
                                        String contactId_string = String.valueOf(contactId);
                                        long timestamp = cursor.getLong(4);

                                        String body = cursor.getString(5);

                                        if (!unreadOnly) {
                                                count = 0;
                                        }

                                        SmsMmsMessage smsMessage = new SmsMmsMessage(
                                                        context, address, contactId_string, body, timestamp,
                                                        threadId, count, messageId, SmsMmsMessage.MESSAGE_TYPE_SMS);

                                        return smsMessage;

                                }
                        } finally {
                                cursor.close();
                        }
                }               
                return null;
        }

How to use Union method or left outer join?

i can not join #Temp with scr_SecuristLog. How can i do it?

CREATE TABLE #Temp (VisitingCount int, [Time] int )
 DECLARE @DateNow DATETIME,@i int,@Time int
    set @DateNow='00:00'  
    set @i=1;  
    while(@i<48)  
        begin  
set @DateNow = DATEADD(minute, 30, @DateNow)
set @Time = (datepart(hour,@DateNow)*60+datepart(minute,@DateNow))/30 
insert into #Temp(VisitingCount,[Time]) values(0,@Time )
set @i=@i+1
                end
select VisitingCount, [Time]
from #Temp as t
left outer join (
    select count(page) as VisitingCount, 
    (datepart(hour,Date)*60+datepart(minute,Date))/30 as [Time]
    from scr_SecuristLog
    where Date between '2009-05-04' and '2009-05-05'
) as s
    on t.VisitingCount = s.VisitingCount
        and t.Time = s.Time

This Codes give error:

Msg 8120, Level 16, State 1, Line 1 Column 'scr_SecuristLog.Date' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.
Msg 8120, Level 16, State 1, Line 1 Column 'scr_SecuristLog.Date' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.
Msg 209, Level 16, State 1, Line 1 Ambiguous column name 'VisitingCount'.
Msg 209, Level 16, State 1, Line 1 Ambiguous column name 'Time'.

From stackoverflow

As you don't mention a specific error, I am guessing your error comes from the fact you have not prefixed your select values.

select t.VisitingCount, t.[Time]

Edit

Your second error should be resolved with this group by.

select count(page) as VisitingCount, 
(datepart(hour,Date)*60+datepart(minute,Date))/30 as [Time]
from scr_SecuristLog
where Date between '2009-05-04' and '2009-05-05'
GROUP BY (datepart(hour,Date)*60+datepart(minute,Date))/30

I think you need to add a GROUP BY in your derived table scr_SecuristLog, you need to group it by time because your using an aggregate function count.

Ok try this

CREATE TABLE #Temp (VisitingCount int, [Time] int )
DECLARE @DateNow DATETIME,@i int,@Time int
set @DateNow='00:00'  
set @i=1;  
while(@i<48)  
    begin  
        set @DateNow = DATEADD(minute, 30, @DateNow)
        set @Time = (datepart(hour,@DateNow)*60+datepart(minute,@DateNow))/30 
        insert into #Temp(VisitingCount,[Time]) values(0,@Time )
        set @i=@i+1
    end
    select t.VisitingCount, t.[Time]
    from #Temp as t
    left outer join (
         select count(page) as VisitingCount, 
       (datepart(hour,Date)*60+datepart(minute,Date))/30 as [Time]
       from scr_SecuristLog
       where Date between '2009-05-04' and '2009-05-05'
       GROUP BY scr_SecuristLog.Date
   ) as s
    on t.VisitingCount = s.VisitingCount
    and t.Time = s.Time

Your inner select (which you are joining on) does not aggregate properly. You have a count and a column. This means you need to group by the column for SQL to understand it properly.

select t.VisitingCount, t.[Time]
from #Temp as t
left outer join (
    select count(page) as VisitingCount, 
    (datepart(hour,Date)*60+datepart(minute,Date))/30 as [Time]
    from scr_SecuristLog
    where Date between '2009-05-04' and '2009-05-05'
    GROUP BY [Date]
) as s
    on t.VisitingCount = s.VisitingCount
        and t.Time = s.Time

What you might actually need is to group by the calculated column instead, in which case your group by should be:

GROUP BY (datepart(hour,Date)*60+datepart(minute,Date))/30


CREATE TABLE #Temp (
  VisitingCount INT,
  [Time] INT)
DECLARE @DateNow DATETIME,
  @i INT,
  @Time INT
SET @DateNow = '00:00'

SET @i = 1 ;

WHILE(@i < 48)

  BEGIN

    SET @DateNow = DATEADD(minute, 30, @DateNow)
    SET @Time = (DATEPART(hour, @DateNow) * 60 + DATEPART(minute, @DateNow))
      / 30 
    INSERT  INTO #Temp (VisitingCount, [Time])
    VALUES  (0, @Time)
    SET @i = @i + 1
  END
SELECT  VisitingCount,
        [Time]
FROM    #Temp AS t
UNION
SELECT  COUNT(page) AS VisitingCount,
        (DATEPART(hour, Date) * 60 + DATEPART(minute, Date)) / 30 AS [Time]
FROM    scr_SecuristLog
WHERE   Date BETWEEN '2009-05-04' AND '2009-05-05'
GROUP BY Date

DROP TABLE #Temp

How to configure WCF end-to-end logging?

Hi, I have a problem that I'd like to solve, but I don't know how to configure end-to-end logging in WCF (I'm not sure I can solve my problem even with end-to-end logging enabled). I have three applications (A, B, C) that use WCF service through NetMsmqBinding to communicate with each other. A is the host, and B and C are clients of the WCF service. The service uses message security with certificates. The communication between B and A works perfectly OK, though, the communication between C and A does not. So I'd like to use logging and figure out why the messages are deleted from the queue (in the case C -> A) and are not processed by the host... How do we configure logging for this scenario in order to diagnose the problem... (app B and app C are on the same machine, app A is on a different machine). (B -> A works both with and without certificates) (C -> A works ONLY without certificates). I need to configure C -> A to work with certificates, but I don't know what's wrong in order to fix it. Please help!

Thanks!

From stackoverflow

Here is how to do WCF end-to-end tracing:

http://msdn.microsoft.com/en-us/library/aa738749.aspx

In your case I would have used Enterprise Library Logging. Logged entry and exit of each method with which parameters were sent, as well as using try catch around critical parts of the code, with logging of the exception.

Hope this helps.

Design patterns for web services?

Hi,

I am developing web service in C#, .NET 2.0. What are the proven design patterns for a web service? If possible please provide me the link about these design patterns?

From stackoverflow

A few links:
- Principles of Service Design: Service Patterns and Anti-Patterns
- Applying Design Issues and Patterns in Web Services
Also look at Service Factory. http://www.codeplex.com/servicefactory http://msdn.microsoft.com/en-us/magazine/cc163514.aspx is overview article about that.

alextansc : I seconded this. I started using it about 2 years ago, and still reaping many benefits from the initial time invested on it ever since.
Hi there,

Here is a link to an example using the begin/end design pattern with web-services. I wont go into detail as the article is pretty concise. Hope this helps http://www.codeproject.com/KB/cpp/wsasync.aspx
Do model view controller and RESTful interfaces.

Nagaraj : Can i get More details about this.if possible any links.

tmadsen : www.asp.net/mvc http://blog.wekeroad.com/blog/aspnet-mvc-using-restful-architecture/

C# mono p/invoke failure

Recently I tried to use p/invoke in mono (.NET for Linux platform) to write a simple OpenGL application to find out how it works on C# (I've already sucessfully done it on windows). I heard about the tao framework, but I don't want everything for a simple "hello world" like program.

I just stucked at the start. I p/invoked some GL functions to see if they work. I immediately called glClearColor and glClear to see whether it sets glGetError or not (due to opengl haven't been initialized at all at that point).

Instead of calling the function it just crashes and dumps the following stack trace and other debuginfo. No exception is thrown.

Stacktrace:

  at (wrapper managed-to-native) Calmarius.OGL.OpenGLLibrary.glClearColor (single,single,single,single) <0x00004>
  at (wrapper managed-to-native) Calmarius.OGL.OpenGLLibrary.glClearColor (single,single,single,single) <0xffffffff>
  at Calmarius.RTS.GameForm.OnPaint (System.Windows.Forms.PaintEventArgs) [0x00000] in /home/calmarius/Development/csharp/RTS/RTS/Form1.cs:60
  at System.Windows.Forms.Control.WmPaint (System.Windows.Forms.Message&) <0x000b0>
  at System.Windows.Forms.Control.WndProc (System.Windows.Forms.Message&) <0x001e2>
  at System.Windows.Forms.ScrollableControl.WndProc (System.Windows.Forms.Message&) <0x0000d>
  at System.Windows.Forms.ContainerControl.WndProc (System.Windows.Forms.Message&) <0x00054>
  at System.Windows.Forms.Form.WndProc (System.Windows.Forms.Message&) <0x001da>
  at ControlWindowTarget.OnMessage (System.Windows.Forms.Message&) <0x00014>
  at ControlNativeWindow.WndProc (System.Windows.Forms.Message&) <0x00022>
  at System.Windows.Forms.NativeWindow.WndProc (intptr,System.Windows.Forms.Msg,intptr,intptr) <0x001b7>
  at System.Windows.Forms.XplatUIX11.DispatchMessage (System.Windows.Forms.MSG&) <0x00016>
  at System.Windows.Forms.XplatUI.DispatchMessage (System.Windows.Forms.MSG&) <0x00015>
  at System.Windows.Forms.Application.RunLoop (bool,System.Windows.Forms.ApplicationContext) <0x00997>
  at System.Windows.Forms.Application.Run (System.Windows.Forms.ApplicationContext) <0x0006a>
  at System.Windows.Forms.Application.Run (System.Windows.Forms.Form) <0x00025>
  at Calmarius.RTS.Program.Main () [0x0000b] in /home/calmarius/Development/csharp/RTS/RTS/Program.cs:19
  at (wrapper runtime-invoke) System.Object.runtime_invoke_void (object,intptr,intptr,intptr) <0xffffffff>

The signature for glClearColor is:

//gllibname="opengl32.dll" --> mapped to libGL.so
[DllImport(gllibname)]

public static extern void glClearColor(float red, float green, float blue, float alpha);

C specification is:

void glClearColor( GLclampf red,GLclampf green,GLclampf blue,GLclampf alpha );

GLclampf is float as I saw its declaration in the header.

From stackoverflow

You could run the program with gdb and see exactly where the SEGV happens (see the mono wiki for instructions).

A likely cause is that some other incorrect p/invoke declaration and call in the code corrupted memory so later you get the crash.

I want a way to know if a particular web page is completely loaded using javascript

Hi,

I want a way to know if a particular web page is completely loaded using javascript

Ex) A web page can have dynamic contents getting displayed through Ajax, or contents comming through javascript, all the images and css.I need to find out when all these contents are loaded properly and the webpage is stablilized.

I got some pointers like ondocumentready its an alternate soultion for window.onload, onreadystatechange but these solutions are unreliable sometimes it works and sometimes it doesn’t

Also the solution should be browser independent, i.e it should work for all browsers

Any pointers on this will be highly appreciated

Thanks in advance

From stackoverflow

Use jQuery and its easy:

<script type="text/javascript">
    $(document).ready(function() {
        alert('doc is ready');
        });
    });
</script>

browser independent
highly reliable
awesome

You can also use the following. It's a shorthand version of the code posted by @cottsak
```
$(function() 
{ 
     // your code 
});
```
if you’re using jQuery.noConflict() (which is used to give control of the $ variable back to whichever library first implemented it), you can do this:

By adding jQuery.noConflict(), jquery won't conflict with $ variables of other libraries
```
jQuery(function($) 
{ 
       // your code
});
```
Using pure javascript you can put javascript in the onload event of the body tag like so:
```
<body onload="function">
```
This will trigger when the page has finished loading.

Separation of responsibility in assemblies

Hi All, We are working on an Asp.net project. It has two parts, first one is for the public users (to be used on internet) and other one is for internal use inside the organization. The processes and objects are more complicated, vast and intricate on the intranet side.

Though both the sites will be using the same database (to be connected with using DAL generated by SubSonic) so there can be same library for both but the way they manipulate the objects and business logic involved is very different on both sides. Now my dilemma is:

1) to keep bll files in separate namespaces like (public.bll, internal.bll) and different projects (mind the responsibilities and business logic is different for both)
or
2) keep all the blls in one namespace (project.bll)

(bll: Business Logic Layer, as used here in Visual Studio)

From stackoverflow

I think you should consider a third option - splitting into three groups:

1) public.bll - all the logic that is specific only to the public area.
2) internal.bll - all the logic that is specific only to the internal area.
3) shared.bll - all the logic that both areas will use, for example the DAL.

You then include shared.bll in both projects.

The user is not associated with a trusted SQL Server connection.

i wrote the code like this to connect sqlserver database.i load the driverclass,but iam not getting connection. Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver"); System.out.println("class Loaded");
connection = DriverManager.getConnection("jdbc:sqlserver://10.171.160.114:2001","XXXXXX","XXXXXX");
System.out.println("connected");

iam getting this error " The user is not associated with a trusted SQL Server connection. ".please advice me on this.any help greately appreciated.

From stackoverflow

It probably has to do with the authentication mechanism you use. See this question for details.

For more information, read here.
The SQL server has been configured to operate in "Windows Authentication Mode (Windows Authentication)" and doesn't allow the use of SQL accounts. Right click on your SQL Server node in Management Studio and choose Properties. On the dialog that appears go to security tab and make sure that "SQL Server and Windows Authentication mode" is selected

VB6.0 : initialize method of a User Control called when loading a VB project.

Whenever we load a VB project it will call initialize method of a User Control ( if there is any in the project). My problem is that is that I have some code in UserControl.initialize that will try to create instances of other COM objects. And on my build machine those controls are not registered. One option is to move the code to some other method other than initialize but I want to know if there is a better solution? Somewhere I found that we may have a check to see if the calling application is VB editor then skip the initialization code...

From stackoverflow

This only happens if your project was saved with the form designer open: this means that at startup the form is displayed (maybe in the background) and consequently, all controls on it need to be initialized. Hence, your user control initializer is called if this control is used on the form.

To prevent this, simply save the project with the form designer closed.

Paragon : Yup you are right. But my problem is that I can't make sure that each of my developer checks in the project file with the designer closed :(
You can use:
```
If Not Me.DesignMode Then
  ...
End If
```
An other solution we used was a little function which can be used globally:
```
Public Function IsRuntime() as Boolean
   On Error Goto NotRuntime
   Debug.Print(1 / 0)
   IsRuntime = True
   Exit Function
NotRuntime:
   IsRuntime = False
End If
```
Don't know if it is syntactically well formed, but the idea should be clear: Only in the IDE the debug statement is called.

Paragon : Thanks H-Man2 :). The second solution is great. As for the first solution Me.DesignMode is not an option in VB6, perhaps it was introduced in VB.NET

MarkJ : The first solution should be If Not Ambient.UserMode for VB6. For example http://msdn.microsoft.com/en-us/library/aa241637(VS.60).aspx

MarkJ : Note that the two solutions have slightly different behaviour. The first solution always switches off the code when the control is in design-mode (only makes a difference if you build your control into an OCX). The second solution always switches the code off when running from the IDE whether in design mode *or in run mode*. Might stop your control working properly when running from the IDE to test run-time behaviour.

ASP.net MVC RTM - "project type is not supported"

After installing ASP.net MVC, when I attempt to load a existing or a new MVC application (straight from New Project - A project for creating an application using the ASP.NET MVC framework (1.0) (.NET Framework 3.5)) I get the following error:

Microsoft Visual Studio

The project file 'path' cannot be opened.

The project type is not supported by this installation.

OK Help

Tried reinstalling which didn't help. Tried devenv /setup which also didn't help.

Any ideas?

Thanks

Ben

From stackoverflow

Not sure if this is the same issue but going through the uninstall and re-install of beta and RC versions and finally the RTW I ended up with two ASP.NET MVC Web Application icons in my "File/New/Project" dialog.

If I click the first link, my project does not load correctly. But if I click the second one everythign works great.

I haven't dug into the templates folder to see if there is a remnant from a previous version or what.

Brian

Ben Hall : Hi Brain, thanks for that. I only have the one template, this laptop only ever had RTW installed on it.
Did you install SP1 for .Net and VS2008?
You could also try to run the following command:
```
devenv /ResetSkipPkgs
```
This will try to load any Visual Studio packages that failed previously at some point, such as the WPF project flavor package, which would cause the error message you're seeing.
The reason was because I didn't have the Visual Web Development part of VS installed. I only had the C# language. As such, MVC didn't check and installed successfully but couldn't use the underlying part.

After installing the Web Dev bit everything worked as expected...

Information Rights Management (IRM) & SharePoint - can the system administrators view the content of a protected file?

We are considering deploying a MOSS site for our HR department but there is some concern over the system administrators being able to access the highly confidential material regarding performance and salaries. It appears that IRM is the silver bullet we are looking for providing the MOSS System Administrators CAN NOT open the documents to view the content.

Can anyone confirm that IRM will secure the content from our Sys Admins in our MOSS Site?

Thanks! -Tim

From stackoverflow

There are 2 ways in using IRM with SharePoint:
1. SharePoint integrated mode: In this mode, IRM is applied on the document on the fly with correct permission when the user download it. In this case the document is stored unprotected, so administrator with direct access to the SQL database can view the content of the document.
2. "Standard": If IRM is applied on the document before uploading it to SharePoint, the document is stored protected (there are 2 protections: SharePoint permissions AND IRM permissions). So even SQL administrator won't be able to extract the document in this case. However, there are some side effects: for example search won't work as the search engine won't be able to extract the content of the document.
In your case, you will need to use the second solution: encrypt the documents before uploading them to SharePoint.
I'm interested in the first way of using IRM with Sharepoint: 'The Sharepoint Integrated Mode'. But the problem is that when I enable IRM on a document library by creating a Policy, and then I try to open a word document (an un-restricted document) from document library after uploading it in the document library, the document doesn't open and I'm left with a blank Microsoft Word window. This doesn't happen with I try to open a restricted document from document library (by restricted document I mean that some permissions are assigned to it from within Microsoft Word before uploading it to the document library). Can you tell me why I'm not able to view an unprotected Word document within an IRM enabled document library?
Have you looked at Adeptol Rights management for sharepoint at www.adeptol.com

Python -- Regex -- How to find a string between two sets of strings

Consider the following:

<div id=hotlinklist>
  <a href="foo1.com">Foo1</a>
  <div id=hotlink>
    <a href="/">Home</a>
  </div>
  <div id=hotlink>
    <a href="/extract">Extract</a>
  </div>
  <div id=hotlink>
    <a href="/sitemap">Sitemap</a>
  </div>
</div>

How would you go about taking out the sitemap line with regex in python?

<a href="/sitemap">Sitemap</a>

The following can be used to pull out the anchor tags.

'/<a(.*?)a>/i'

However, there are multiple anchor tags. Also there are multiple hotlink(s) so we can't really use them either?

From stackoverflow

Don't use a regex. Use BeautfulSoup, an HTML parser.

from BeautifulSoup import BeautifulSoup

html = \
"""
<div id=hotlinklist>
  <a href="foo1.com">Foo1</a>
  <div id=hotlink>
    <a href="/">Home</a>
  </div>
  <div id=hotlink>
    <a href="/extract">Extract</a>
  </div>
  <div id=hotlink>
    <a href="/sitemap">Sitemap</a>
  </div>
</div>"""

soup = BeautifulSoup(html)
soup.findAll("div",id="hotlink")[2].a

# <a href="/sitemap">Sitemap</a>

Parsing HTML with regular expression is a bad idea!

Think about the following piece of html

<a></a > <!-- legal html, but won't pass your regex -->

<a href="/sitemap">Sitemap<!-- proof that a>b iff ab>1 --></a>

There are many more such examples. Regular expressions are good for many things, but not for parsing HTML.

You should consider using Beautiful Soup python HTML parser.

Anyhow, a ad-hoc solution using regex is

import re

data = """
<div id=hotlinklist>
  <a href="foo1.com">Foo1</a>
  <div id=hotlink>
    <a href="/">Home</a>
  </div>
  <div id=hotlink>
    <a href="/extract">Extract</a>
  </div>
  <div id=hotlink>
    <a href="/sitemap">Sitemap</a>
  </div>
</div>
"""

e = re.compile('<a *[^>]*>.*</a *>')

print e.findall(data)

Output:

>>> e.findall(data)
['<a href="foo1.com">Foo1</a>', '<a href="/">Home</a>', '<a href="/extract">Extract</a>', '<a href="/sitemap">Sitemap</a>']

Ben Blank : If you replace that `.*` with `(?:[^<]+|<(!/a\b))*`, you'll get fewer false positives, without blowing up the regex engine with backtracking.

Use BeautifulSoup or lxml if you need to parse HTML.

Also, what is it that you really need to do? Find the last link? Find the third link? Find the link that points to /sitemap? It's unclear from you question. What do you need to do with the data?

If you really have to use regular expressions, have a look at findall.
In order to extract the contents of the tagline:
```
    <a href="/sitemap">Sitemap</a>
```
... I would use:
```
    >>> import re
    >>> s = '''
    <div id=hotlinklist>
    <a href="foo1.com">Foo1</a>
      <div id=hotlink>
        <a href="/">Home</a>
      </div>
      <div id=hotlink>
        <a href="/extract">Extract</a>
      </div>
      <div id=hotlink>
        <a href="/sitemap">Sitemap</a>
      </div>
    </div>'''
    >>> m = re.compile(r'<a href="/sitemap">(.*?)</a>').search(s)
    >>> m.group(1)
    'Sitemap'
```
VN44CA : Actually, replace sitemap with XYZ as it really can be anything. I would only know that it is the 3rd div within the hotlinlist div. The html pattern that used can be repeated many times. Let say I want to take out all the smart phones listing on ebay. I would know that the above pattern is repeated for each smart phone found, however, the XYZ can be an iphone, blackberry, Nokia or any other smart phone. There could be no item or 100s. So, I was looking for something that says find the repeated pattern, then take the smart phone line out and have a list of smart phones.

Using ctypes.c_void_p as an input to glTexImage2D?

I'm using a 3rd party DLL to load in some raw image data, and I want to use this raw image data as a texture in openGL. However, the c function returns a void*, and I need to somehow convert this so it will work as the "pixels" parameter to glTexImage2D. Right now my code looks like something this:

data = c_void_p(vdll.vlImageGetData()) 
glTexImage2D(GL_TEXTURE_2D, 1, GL_RGB8, 1024, 1024, 0, GL_RGB, GL_UNSIGNED_BYTE, data )

However, I get a TypeError complaining that data 'cannot be converted to pointer'. Does anyone know how to get this to work?

Edit: Figured it out. Basically what I do is this:

data = create_string_buffer( BUFFER_SIZE )
data = dll.vlImageGetData()
glTexImage2D(GL_TEXTURE_2D, 1, GL_RGB8, 1024, 1024, 0, GL_RGB, GL_UNSIGNED_BYTE, data )

From stackoverflow

An answer to a similar question suggested to use ctypes.cast().

aeflash : This doesn't work because the pyopenGL expects the pixels param to be a raw string. I can't use c_char_p because that expects a null-terminated string. I need a way to convert that void pointer into a raw python string.

wr : Have you already seen http://osdir.com/ml/python.ctypes/2004-12/msg00002.html ? If yes and it didn't help, you should provide a small stand-alone skript that we can execute locally.

Useful tools for working with Subversion

There are a number of great tools out there which integrate with Subversion and do some really neat things. I’ve tried a number of them but ultimately only tend to use the following on a regular basis:

Tortoise (the ubiquitous Explorer plugin)
Visual SVN (IDE integration for VS)
Visual SVN Server (light weight SVN server)
SVN Monitor (repository monitoring)

What else is out there that you find really innovative or useful for interacting with Subversion?

From stackoverflow

I just end up using Subclipse just to stay in my IDE.
Coupling integration servers with version control does wonders to your build process. Two of my favorites are CruiseControl.NET and Hudson - but there's lots out there. Note that both of these work seamlessly with SVN.
Some kind of subversion web access apps. You can find an comparison here: http://geekswithblogs.net/flanakin/articles/CompareSubversionWebTools.aspx

Jimmie R. Houts : Thats a pretty old comparison :)
I really like SyncroSVN (I'm using it on a Mac) Easy to use, built in dif and SSH support.
I use Commit Monitor to get notified of commits made by other devs.

I'll have to take a look at SVN Monitor, looks like it does the same thing, but has more features than Commit Monitor.

Evan : Yeah, CM is okay, but I ended up just putting in a post-commit hook to send emails instead. Saves having yet another program on my computer, and saves the server being polled all the time.
I agree with Peter D and Gabriel Florit; in Eclipse you get a gui and you can easily move around in the svn repository, deleting or renaming or whatever. And with Hudson I have it check out my projects and build them and run their unit tests.

I haven't really explored it, but Maven has plugins for subversion. I use Maven for building and deploying but haven't used it with the subversion plugin.
In addition to some previous comments:
- TeamCity has excellent support for Subversion in terms of build tools other than CC.NEt and Hudson.
- AnkhSVN with Visual Studio (it's latest builds look quite nice)
- Fisheye from Atlassian is an awesome way to monitor and view your source code/history/revisions with some neat statistics and support for code reviews, etc.
StatSVN is a really cool, albeit someone nerdy, tool for reporting details about your repo and commit statistics; it's really just graph porn :)

Here are some examples of the graphs.
I have used Synchro on Windows and Mac. It works well especially if you have developers on different platforms. I think that the license allows you a single user to use it on multiple platforms (I do full time development on Mac and test on Linux and Windows).

Versions is a Mac client with a nice UI.
I use kdesvn as a linux client, does the job.
Trac is a issue tracker with SVN integration. I really like the timeline view, where one can quickly see all the changesets for quick code reviews.
I think some of the hook scripts that are available are quite useful to set up on your repository. For instance I use a pre-commit hook to ensure a minimum log message length (some of our devs were prone to committing with an empty log message) and a post-commit hook to send an email to a mailing list.

I also think RepoGuard looks pretty interesting, but haven't had a chance to use it yet.
the svn protocol handler is also quite useful: it allows you to browse svn:// repositories in internet explorer.
I would add a normal Subversion commandline client (e.g. from SlikSvn). By far the easiest way to script and/or automate subversion commands.

And maybe SharpSvn if you need more scripting power or when you would like to integrate Subversion support in your own .Net application. (AnkhSVN, Svn Monitor and several other tools are built on top of the SharpSvn binding).
On Mac, I like Versions. Other people prefer Cornerstone. Both are nice SVN clients with excellent UI.
SVNPlot generate cool statistics from your Subversion repository

Nitin Bhide : SVNPlot - http://code.google.com/p/svnplot
When writing or editing text in Microsoft Office, I stay focused on my task. Like most of people, I feel disturbed when I have to interrupt my work for some maintenance. Nevertheless, keeping document's change history under Subversion can be considered a good practice.

For comparison, one must make as many as 5 steps in order to update a document via TortoiseSVN:
1. close the document;
2. open its folder in Windows Explorer;
3. execute an Update command via context menu,
4. wait for its completion, close TortoiseSVN window,
5. open the document again.
I can reduce all these steps to just one click by using MagnetSVN add-in for Microsoft Office.
It is a real time saver!
Winmerge is a fairly nice Diff tool that integrates well with tortoise svn. I like it better than the default diff tool that comes with tortoise.

Mysql connectivity from POcket pc application using .netcf 2.0

Hi, We are developing an order taking system in PDA using C# in .netcf 2.0. So we are having mysql server as our back end. Is that the web service is the only way to connect mysql with PDA? Or is any other methods available..?

From stackoverflow

I don't believe there is a Data Provider for MySQL in Compact Framework.

I recommend that you read the Mobile Application Architecture Guide, especially the Data Layer chapter to see what your options are.
There is a .netcf connector on MySql web site available for download. I recommend to use version 5.2.7 and not to forget to put "pooling=false" into the connection string, otherwise connection will fail with exception "MissingManifestResourceException" .

Connection string example: "Persist Security Info=False;database=DATABASE_NAME;server=SERVER_IP_ADDRESS;user id=USERNAME_TO_LOGON;Password=PASSWORD;pooling=false"

Obtain a switch/case behaviour in Perl 5

Is there a neat way of making a case or switch statement in Perl 5?. It seems to me they should include a switch on version 6..

I need this control structure in a script, and I've heard you can import a "switch module". But how can I achieve it without imports to minimize dependencies and acquire portability?

From stackoverflow

If you are using Perl 5.10 you have given/when which is a switch statement (note, it can do more than compare with regexes, read the linked docs to see its full potential):

#or any of the dozen other ways to tell 5.10 to use its new features
use feature qw/switch/; 

given($string) {
    when (/^abc/) { $abc     = 1; }
    when (/^def/) { $def     = 1; }
    when (/^xyz/) { $xyz     = 1; }
    default       { $nothing = 1; }
}

If you are using Perl 5.8 or earlier you must make do with if/elsif/else statements:

if    ($string =~ /^abc/) { $abc     = 1; }
elsif ($string =~ /^def/) { $def     = 1; }
elsif ($string =~ /^zyz/) { $xyz     = 1; }
else                      { $nothing = 1; }

or nested condition operators (?:):

$string =~ /^abc/ ? $abc     = 1  :
$string =~ /^def/ ? $def     = 1  :
$string =~ /^xyz/ ? $xyz     = 1  :
                    $nothing = 1;

There is a module in Core Perl (Switch) that gives you fake switch statements via source filters, but it is my understanding that it is fragile:

use Switch;

switch ($string) {
    case /^abc/ {
    case /^abc/ { $abc     = 1 }
    case /^def/ { $def     = 1 }
    case /^xyz/ { $xyz     = 1 } 
    else        { $nothing = 1 }
}

or the alternate syntax

use Switch 'Perl6';

given ($string) {  
    when /^abc/ { $abc     = 1; }
    when /^def/ { $def     = 1; }
    when /^xyz/ { $xyz     = 1; }
    default     { $nothing = 1; }
}

Brad Gilbert : The Perl 5.10 version sets $_ like it does in Perl 6. the Switch module's given/when doesn't.

Alexandr Ciornii : 'when' is an implicit '~~' operator, so not only regexes are allowed.

The suggestion in Programming Perl is:


for ($string) {
    /abc/ and do {$abc    = 1; last;};
    /def/ and do {$def    = 1; last;};
    /xyz/ and do {$xyz    = 1; last;};
    $nothing = 1;
}

An equivalent solution that I like is a dispatch table.

my $switch = {
  'case1' => sub { print "case1"; },
  'case2' => sub { print "case2"; },
  'default' => sub { print "unrecognized"; }
};
$switch->{$case} ? $switch->{$case}->() : $switch->{'default'}->();

ephemient : Shorter: `($switch->{$case} || $switch->{default})->()`

Just a short comment about the core Switch module that's been mentioned a couple of times in answers. The module in question relies on source filters. Among other things, that may result in wrong lines reported for errors. It's so bad that none of the core developers really remembers or cares to remember why it was accepted into the perl core in the first place.

Furthermore, Switch.pm will be the first Perl module ever to be removed from the perl core. The next major release of perl, 5.12.0, will still have it, albeit with a deprecation warning. That deprecation warning will go away if you explicitly install Switch.pm from CPAN. (You get what you ask for.) In the next release down the road, 5.14, Switch.pm will be entirely removed from core.

Java Deserialization of java.lang.Integer - Exception

Recieved the following exception when deserializing a HashMap<String, Integer>:

java.io.InvalidClassException: java.lang.Integer; local class incompatible: stream classdesc serialVersionUID = 1360826667802527544, local class serialVersionUID = 1360826667806852920

Serialized and deserialized on the same machine, with the same JRE. JDK 1.6.0_12

From stackoverflow

Hello, check the source code for Integer, here is what I have for Integer in several verions of java:
```
/** use serialVersionUID from JDK 1.0.2 for interoperability */
private static final long serialVersionUID = 1360826667806852920L;
```
So I'd say the problem comes from a class of yours that you changed between serialization and deserialization and that has no specific serialVersionUID...

Maybe you should look at this, same problem description and it looks like wrong serialization / deserialization code....

harto : It's weird, because the first bunch of digits in the serialized Integer are the same (136082666780...) - which suggests something weird has happened to the serialized file, or perhaps the input stream(?)

Matthew Flaschen : OpenJDK 7 (http://hg.openjdk.java.net/icedtea/jdk7/jdk/file/65dfd94650d3/src/share/classes/java/lang/Integer.java) still uses this ID, which is also the ID identified as the "local class". So somehow it looks like you're deserializing a non-standard/incompatible Integer.

pgras : your're right no answer at the momment but if you google 'serialVersionUID = 1360826667802527544' you'll see he is not alone :)

Matthew Flaschen : harto has a point. It could just be good old fashioned file corruption.
That shouldn't happen. Note that the IDs differ only in the last few digits; the second one ist the one I see in my JDK sources.

My guess is that the serialized stream got corrupted somehow.
From looking at the JDK source, 1360826667806852920 is the correct serialVersionUID for Integer. I wasn't able to find any classes in the JDK with the serialVersionUID 1360826667802527544.

Interestingly, searching for 1360826667802527544 on Google turned up a few other people with this problem, notably this thread on Sun's forums. The problem there was that the person was storing bytes in a String, and the serialized data was getting mangled. Since you're getting the same serialVersionUID it seems very likely that you're running into a similar problem.

Never store bytes in a String. Use a byte array or a class designed to hold bytes, not chars.

Clearing Python lists

Is there any reason to do anything more complicated than one of these two lines when you want to clear a list in Python:

old_list = []
old_list = list()

The reason I ask is that I just saw this in some running code:

del old_list[ 0:len(old_list) ]

From stackoverflow

To clear the list in place, so references "see" the same list.

For example, this method doesn't clear the referenced list:
```
>>> a = [1,2,3]
>>> b = a
>>> a = []
>>> print a
[]
>>> print b
[1, 2, 3]
```
But this one does:
```
>>> a = [1,2,3]
>>> b = a
>>> del a[0:len(a)]
>>> print a
[]
>>> print b
[]
>>> a is b
True
```
You could also do:
```
>>> a[:] = []
```
: Good point, thanks!

Renato Besen : You have a good point

musicfreak : Hmm, you learn something new every day.
~~It appears to me that del will give you the memory back, while assigning a new list will make the old one be deleted only when the gc runs.matter.~~

This may be useful for large lists, but for small list it should be negligible.

Edit: As Algorias, it doesn't matter.

Note that
```
del old_list[ 0:len(old_list) ]
```
is equivalent to
```
del old_list[:]
```
Algorias : gc is not run in cycles. The list is freed as soon as its last reference is dropped.
There are two cases in which you might want to clear a list:
1. You want to use the name old_list further in your code;
2. You want the old list to be garbage collected as soon as possible to free some memory;
In case 1 you just go on with the assigment:
```
    old_list = []    # or whatever you want it to be equal to
```
In case 2 the del statement would reduce the reference count to the list object the name old list points at. If the list object is only pointed by the name old_list at, the reference count would be 0, and the object would be freed for garbage collection.
```
    del old_list
```
You could also do:
```
while l:
    l.pop()
```
FogleBird : LOL. I'm sure that's efficient.

RoadieRich : Nobody said anything asbout it being efficient...

Oleksandr Bolotov : +1 for evillness

Is a graph library (eg NetworkX) the right solution for my Python problem?

I'm rewriting a data-driven legacy application in Python. One of the primary tables is referred to as a "graph table", and does appear to be a directed graph, so I was exploring the NetworkX package to see whether it would make sense to use it for the graph table manipulations, and really implement it as a graph rather than a complicated set of arrays.

However I'm starting to wonder whether the way we use this table is poorly suited for an actual graph manipulation library. Most of the NetworkX functionality seems to be oriented towards characterizing the graph itself in some way, determining shortest distance between two nodes, and things like that. None of that is relevant to my application.

I'm hoping if I can describe the actual usage here, someone can advise me whether I'm just missing something -- I've never really worked with graphs before so this is quite possible -- or if I should be exploring some other data structure. (And if so, what would you suggest?)

We use the table primarily to transform a user-supplied string of keywords into an ordered list of components. This constitutes 95% of the use cases; the other 5% are "given a partial keyword string, supply all possible completions" and "generate all possible legal keyword strings". Oh, and validate the graph against malformation.

Here's an edited excerpt of the table. Columns are:

keyword innode outnode component

acs 1 20 clear
default 1 100 clear
noota 20 30 clear
default 20 30 hst_ota
ota 20 30 hst_ota
acs 30 10000 clear
cos 30 11000 clear
sbc 10000 10199 clear
hrc 10000 10150 clear
wfc1 10000 10100 clear
default 10100 10101 clear
default 10101 10130 acs_wfc_im123
f606w 10130 10140 acs_f606w
f550m 10130 10140 acs_f550m
f555w 10130 10140 acs_f555w
default 10140 10300 clear
wfc1 10300 10310 acs_wfc_ebe_win12f
default 10310 10320 acs_wfc_ccd1

Given the keyword string "acs,wfc1,f555w" and this table, the traversal logic is:

Start at node 1; "acs" is in the string, so go to node 20.
None of the presented keywords for node 20 are in the string, so choose the default, pick up hst_ota, and go to node 30.
"acs" is in the string, so go to node 10000.
"wfc1" is in the string, so go to node 10100.
Only one choice; go to node 10101.
Only one choice, so pick up acs_wfc_im123 and go to node 10130.
"f555w" is in the string, so pick up acs_f555w and go to node 10140.
Only one choice, so go to node 10300.
"wfc1" is in the string, so pick up acs_wfc_ebe_win12f and go to node 10310.
Only one choice, so pick up acs_wfc_ccd1 and go to node 10320 -- which doesn't exist, so we're done.

Thus the final list of components is

hst_ota
acs_wfc_im123
acs_f555w
acs_wfc_ebe_win12f
acs_wfc_ccd1

I can make a graph from just the innodes and outnodes of this table, but I couldn't for the life of me figure out how to build in the keyword information that determines which choice to make when faced with multiple possibilities.

Updated to add examples of the other use cases:

Given a string "acs", return ("hrc","wfc1") as possible legal next choices
Given a string "acs, wfc1, foo", raise an exception due to an unused keyword
Return all possible legal strings:
- cos
- acs, hrc
- acs, wfc1, f606w
- acs, wfc1, f550m
- acs, wfc1, f555w
Validate that all nodes can be reached and that there are no loops.

I can tweak Alex's solution for the first two of these, but I don't see how to do it for the last two.

From stackoverflow

Definitely not suitable for general purpose graph libraries (whatever you're supposed to do if more than one of the words meaningful in a node is in the input string -- is that an error? -- or if none does and there is no default for the node, as for node 30 in the example you supply). Just write the table as a dict from node to tuple (default stuff, dict from word to specific stuff) where each stuff is a tuple (destination, word-to-add) (and use None for the special "word-to-add" clear). So e.g.:
```
tab = {1: (100, None), {'acs': (20, None)}),
       20: ((30, 'hst_ota'), {'ota': (30, 'hst_ota'), 'noota': (30, None)}),
       30: ((None, None), {'acs': (10000,None), 'cos':(11000,None)}),
       etc etc
```
Now handling this table and an input comma-separated string is easy, thanks to set operations -- e.g.:
```
def f(icss):
  kws = set(icss.split(','))
  N = 1
  while N in tab:
    stuff, others = tab[N]
    found = kws & set(others)
    if found:
      # maybe error if len(found) > 1 ?
      stuff = others[found.pop()]
    N, word_to_add = stuff
    if word_to_add is not None:
      print word_to_add
```
Vicki Laidler : Thanks Alex! This is very helpful for the traversal use case. I'm not sure about the other cases though - see updates above. In particular I wonder if a graph library would be applicable in the validation case?

Vicki Laidler : I finally got a chance to work on this project again this week, & realized I never accepted your very helpful answer! Thanks again.

Adding an answer to respond to the further requirements newly edited in...: I still wouldn't go for a general-purpose library. For "all nodes can be reached and there are no loops", simply reasoning in terms of sets (ignoring the triggering keywords) should do: (again untested code, but the general outline should help even if there's some typo &c):

def add_descendants(someset, node):
  "auxiliary function: add all descendants of node to someset"
  stuff, others = tab[node]
  othernode, _ = stuff
  if othernode is not None:
    someset.add(othernode)
  for othernode, _ in others.values():
    if othernode is not None:
      someset.add(othernode)

def islegal():
  "Return bool, message (bool is True for OK tab, False if not OK)"
  # make set of all nodes ever mentioned in the table
  all_nodes = set()
  for node in tab:
    all_nodes.add(node)
    add_desendants(all_nodes, node)

  # check for loops and connectivity
  previously_seen = set()
  currently_seen = set([1])
  while currently_seen:
    node = currently_seen.pop()
    if node in previously_seen:
      return False, "loop involving node %s" % node
    previously_seen.add(node)
    add_descendants(currently_seen, node)

  unreachable = all_nodes - currently_seen
  if unreachable:
    return False, "%d unreachable nodes: %s" % (len(unreachable), unreachable)
  else:
    terminal = currently_seen - set(tab)
    if terminal:
      return True, "%d terminal nodes: %s" % (len(terminal), terminal)
    return True, "Everything hunky-dory"

For the "legal strings" you'll need some other code, but I can't write it for you because I have not yet understood what makes a string legal or otherwise...!

Friday, May 6, 2011

Microsoft Visual Studio

The project type is not supported by this installation.

OK Help

Blog Archive