max_limit hack

A place to discuss development of the Xataface core.

max_limit hack

Postby Cat » Thu Oct 28, 2010 10:31 pm

Hi together,

I'm rather new to Xataface, but seems I got it up and running. Looks like a great app, so congrats to the developers. I've quickly loaded the Wikileaks Iraq War Reports into Xataface, thought Xataface would be like designed for doing so. If someone likes to see how Xataface may look like with the 391832 records inside running on a tiny little box, one may find it here:

http://www.mein-parteibuch.org/iraqwar/

I welcome all ideas to make it better.

What I found "a little bit strange" was that the -limit parameter seems to be neither checked nor can it be restricted, so when a public internet user enters a high value or just an alphabetic char like a in the limit box or into the -limit part of the URI, then the server starts to become very busy for a while and then xataface crashes because it runs out of memory.

It seems to be that someone here in the forum had remarked this behaviour some years before already, too:

Dataface Max Records List?

I don't know, if that problem was addressed before, but I didn't find an option to configure something like a max_limit config parameter out of the box. However a tiny hack into Dataface/Application.php seems to bring this feature almost perfectly. I just added a bit of code just down of line 653 in Dataface/Application.php:

Code: Select all
//this line already exists in Xataface Version 1.25
  if ( !isset( $query['-limit'] )) $query['-limit'] = $this->_conf['default_limit'];

//these lines the Parteibuch Cat addded here to Xataface Version 1.25
   $query['-limit'] = intval(abs($query['-limit']));
   if($query['-limit'] == 0) $query['-limit'] = 1;
   if (isset($this->_conf['max_limit']) && $query['-limit'] > $this->_conf['max_limit']) {
      $query['-limit'] = $this->_conf['max_limit'];
   }


What I did then, is that I put at the top of conf.ini a new directive:

Code: Select all
max_limit = 100


This shall limit the effective value for -limit to integers from 1 to 100 and it seems to work rather fine. So if someone else finds it useful, take the hack code snippet and try yourself. If there is a more elegant way to get a max_limit configuration without hacking inside the xataface app, I would be keen to know about.

PS: What I also found, is that the search and find functions do not behave like I would like it. So far, as a workaround, I disabled the search box because it's consuming too much time on my tiny machine. The find tab I would like to configure in a way, that it searches varchar fields per default like '&search%' instead of '%search%', but I didn't find an elegant way to configure such behaviour, yet. If someone has an idea ...
Last edited by Cat on Thu Oct 28, 2010 11:57 pm, edited 1 time in total.
--
A Cat using Xataface 1.25
Cat
 
Posts: 3
Joined: Thu Oct 28, 2010 8:52 pm

Re: max_limit hack

Postby shannah » Thu Oct 28, 2010 11:57 pm

Very cool. Both your max_limit feature suggestion, and the fact that you used Xataface to publish the famous iraqi war leaks. I'll add your fix to SVN tomorrow so it will be included with the next release. I have generally been coding this sort of thing manually in the beforeHandleRequest hook, but having it as a config option makes a lot of sense.
shannah
 
Posts: 4457
Joined: Wed Dec 31, 1969 5:00 pm

Re: max_limit hack

Postby Cat » Fri Oct 29, 2010 2:44 am

I'm not so sure, that my handling of the situation is the best way. I think it may be that a redirect with a message to the user might be more appropriate.

And I'm neither very sure that I calculated the behaviour of the hack to all possible situations or config options. I found some strange code in ResutController.php at line 341:

Code: Select all
      function limitField($prefix=''){
         $currentLink = Dataface_LinkTool::buildLink(array('-'.$prefix.'limit'=>null));
         if ( !$prefix ) {
            $limitval = $this->_resultSet->limit();
         } else if (isset($_GET['-'.$prefix.'limit'])){
            $limitval = $_GET['-'.$prefix.'limit'];
         } else {
            $limitval = 30;
...


Following line - if used at all - may probably return a strange value if using my hack:

Code: Select all
$limitval = $_GET['-'.$prefix.'limit'];


I'm not sure, how this code is used and just decided, that I don't use prefix, so my quick hack will probably work for me - and some other people with a similar simple config as mine.
--
A Cat using Xataface 1.25
Cat
 
Posts: 3
Joined: Thu Oct 28, 2010 8:52 pm

Re: max_limit hack

Postby shannah » Fri Oct 29, 2010 2:33 pm

I'll look at it more closely before implementing it. The concept is good. The code that you point to isn't too much of a concern because it is subject to whatever settings you placed inside the Application class.

The only thing that you need to watch out for are the export_xml, and export_csv actions which override the limit to a very high number as part of the action. For your purposes, it's best to just disable these actions.

-Steve
shannah
 
Posts: 4457
Joined: Wed Dec 31, 1969 5:00 pm

Re: max_limit hack

Postby shannah » Fri Oct 29, 2010 3:55 pm

A couple of notes about your database. You could improve performance quite a bit by just adding indexes on each of the columns. You would notice a major improvement in performance.

-Steve
shannah
 
Posts: 4457
Joined: Wed Dec 31, 1969 5:00 pm

Re: max_limit hack

Postby Cat » Fri Oct 29, 2010 6:26 pm

Hello Steve,

thanks for your hints.

shannah wrote:... You could improve performance quite a bit by just adding indexes on each of the columns. ...


Of course all columns are indexed and on text columns are fulltext indexes. And for viewing a record identified by key, sorting columns and many search operations it works rather fine. It seems even to use the fulltext index.

Some searches seem to me a bit slow, but I haven't found out yet, where exactly searches are slower than possible. I will give feedback when I found out more.
--
A Cat using Xataface 1.25
Cat
 
Posts: 3
Joined: Thu Oct 28, 2010 8:52 pm


Return to Xataface Developers

Who is online

Users browsing this forum: No registered users and 16 guests

Powered by Dataface
© 2005-2007 Steve Hannah All rights reserved