new user questions and smarty bug

by **webdev** » Fri Jul 17, 2009 10:36 am

Hi,

first of all i wanted to congratulate all the devs on their work on this very nice project.

i've installed xataface 1.1.5r2 on my server to give it a go for a very small project i have to do for a friend. By the way the download link on your home page points to 1.1.2.

I've been reading the forum and documentation/wiki and i'm confident i can customize xataface to match my end user db frontend requirements, although some features are not present out of the box.

First i'd like to get the problem i currently have out of the way so i can continue with my install..
So after configuring a xataface app for my database and poking around with fields.ini, i tried to insert a record in the main (only) table and right after it's inserted i get the following error :

Code: Select all: Warning: Smarty::include(/var/www/lmb/admin/lmb/templates_c/dataface/%%65^65A^65AAB303%%Dataface_Edit_Record.html.php) [smarty.include]: failed to open stream: No such file or directory in /var/www/lmb/admin/lib/Smarty/Smarty.class.php on line 1258 Warning: Smarty::include() [function.include]: Failed opening '/var/www/lmb/admin/lmb/templates_c/dataface/%%65^65A^65AAB303%%Dataface_Edit_Record.html.php' for inclusion (include_path='.:/var/www/lmb/admin:/var/www/lmb/admin/lib:/usr/share/php:/usr/share/pear') in /var/www/lmb/admin/lib/Smarty/Smarty.class.php on line 1258

the path "/var/www/lmb/admin/lmb/templates_c/dataface" exists and there's a bunch of %%...php files there but not the one listed in the error report.

Now for the questions :

- found the answer to my first one there : http://fongwebdev.blogspot.com/2009/05/how-to-hide-field-in-edit-tab.html, in case you wonder if it should be added to your default features in a future release.
For example it would be quite necessary in my opinion to display the "id" (prim key) field of the record on the edit page.

- my second question concerns the import feature which will be used to add records to that database, i've checked out the import filters and i'll probably be able to handle parsing and validating there as expected but i'd need to let the user upload multiple files and was planning on letting them use the following tool http://www.uploadify.com/demo/, and keep a default upload form but with multiple upload fields.

How hard do you think it would be to customize your current upload scheme to make it use what i described above ?
My idea would simply be to store the files on the server and process them one by one using the regular import filter for the current table. All the files would have the same data format, which is not csv but that shouldn't matter.
On failure the filename would be reported using whatever error report you are using.

i have other questions, concerning the text encoding which is broken here for example, but the upload is my main concern at the moment and failure to solve that would make xataface in my case not so helpful.

hope you can help me complete that setup and point me the direction in your core code for that upload problem.

by **webdev** » Sun Jul 19, 2009 3:12 am

I'm probably gonna find a way around for sending a single file instead of multiple files to import records, so unless the modification to support such feature is minimal it should be discarded from my previous message, sorry about that.

I still have that smarty include bug i can't figure out and i'm trying to fix the charset encoding problem i have the pages using a charset iso-8859-1 won't let utf8 database fields be edited properly.

In order to bring your project to a broader audience i'd strongly suggest you switch from your default 8859-1 encoding to utf8 which should now be considered as the default web (and programming) encoding.

by **webdev** » Mon Jul 20, 2009 10:21 am

looks like the smarty template error is gone, must have been some ownership provileges i changed after the initial installation, silly me..

Now i'm having some more serious problems solving my charset encoding errors.

Having a database in UTF-8 (db default and varchar/text fields) i am able to add/edit records properly, meaning that when i check thoses texts in the database they are correctly formated.

But no matter what i do to the conf.ini (ie oe options) and templates, forcing "charset=utf-8", the db texts displayed on the view or in the edit page fields get corrupted.

Another thing i can't quite shake is why xataface refuses to use the oe var to set the html page charset from the template here : charset=<php>_tpl_vars['ENV']['APPLICATION']['oe']; ?>
or maybe somewhere the config var gets overwritten at runtime.

So xataface always sets the html page charset to iso-8859-1 and when i force it to utf8 in the template it doesn't solve the problem.

I tried to revert the database to iso-8859-1 but my tool only offers cp1252 western european collation latin1_general_ci and when i do that and clear the modifications made to xataface files, it still doesn't work and texts disploayed are corrupted (by the browser ?)..

one last thing, when i submit the text again using the corrupted display then it's stored as & #65533;& #65533; (added a space after &) which is supposed to be some html encoding i guess..

i'm quite stuck here, i'll try some other ideas but i'm running late so i hope i can find an answer quickly.

by **shannah** » Mon Jul 20, 2009 12:56 pm

Try using default_oe and default_ie instead of oe and ie:

http://xataface.com/forum/viewtopic.php?t=4256#21382

by **webdev** » Tue Jul 21, 2009 2:39 am

thx it solved the encoding problem.

I changed my database back to utf8 and am using default_ie/oe = utf-8.

by **webdev** » Wed Jul 22, 2009 2:46 pm

A couple of quick questions to continue my customization.

I can't seem to find where or how to put the import filter specific information as described in the default text and in changes.txt "using key convention import_filters:<filter>."

Then i was wondering if there's a way to get the raw value of a TIMESTAMP field, as an integer i guess, in order to avoid converting the string to time as currently done in my delegate_class date display function :

Code: Select all: function d_added__pullValue(&$record, $element){ return date('d-m-Y H:i:s', strtotime($record->strval('d_added'))); }

i've checked the other record methods but couldn't find a raw value.
It's not very important as the above code works just fine.

Finally if the answer is straight enough, i'd like to display the primary key or the record on the edit tab, of course no edit on it, it's currently hidden but nothing is specified in my fields.ini.

btw i've made a few corrections to the french language file and i'll probably continue to update it when i come across some odd expressions, if you're interested in getting my updated file please let me know.

by **webdev** » Thu Jul 23, 2009 3:17 am

okay forget about the "raw integer" timestamp data i was talking above, got confused with some other integer db fields that store unix timestamps used as dates in the php code..

anyways i've been completing my import filter and got a couple of questions i couldn't find answers to when searching the docs :

first, and it's probably a bug, when i simply return an empty "array()" of records, the import filter tells me it has found "1 record" and shows a couple of warnings :

Code: Select all: Warning: array_keys() [function.array-keys]: The first argument should be an array in /var/www/lmb/admin/Dataface/RecordGrid.php on line 99 Warning: Invalid argument supplied for foreach() in /var/www/lmb/admin/Dataface/RecordGrid.php on line 159

the warnings can be fixed by replacing the line 99 :

Code: Select all: from : $columns = array_keys($sampleRecord); to : $columns = ( is_array($sampleRecord) ) ? array_keys($sampleRecord) : array();

but obviously as no records have been returned, i'm not quite sure we should get there in the first place, right ?

It also offers to submit the import what leads to further errors.

Now in your __import__excel_spreadsheet example you do return "array() " when an error occurs in your parser. Does that mean there's no way of returning a message, or updating a global var, to tell the user about what's been done and what has failed ?

In my case i need to do a bit of validating and i'd like to tell which entries have been rejected by the filter, i was also assuming all along that field validators do not apply to import filters..

thx again

by **shannah** » Thu Jul 23, 2009 9:48 am

first, and it's probably a bug, when i simply return an empty "array()" of records, the import filter tells me it has found "1 record" and shows a couple of warnings :

I have added this to the issue tracker. Haven't had a chance to try it yet.
http://bugs.weblite.ca/view.php?id=568

I can't seem to find where or how to put the import filter specific information as described in the default text and in changes.txt "using key convention import_filters:<filter>."

In your language file (e.g. lang/en.ini) you would add a line with:

Code: Select all: import_filters:filter_name = "This is a description of this import filter".

If you don't already have a language file, you just create a directory named 'lang' in your application directory. Then add a file named en.ini (english language file). You could do a french translation by adding fr.ini, etc....

when an error occurs in your parser. Does that mean there's no way of returning a message, or updating a global var, to tell the user about what's been done and what has failed ?

You can return a PEAR_Error (i.e. return PEAR::raiseError("This is the error"). But it may come out a bit ugly. And it wouldn't allow the import to go through smoothly.

You could store a message in the session --msg variable to be displayed on the preview page, if you just wanted to display a warning.

e.g.
$_SESSION['--msg'] = "3 records could not be imported because they contain profanity";

(haven't tested this in a while so let me know if it works for you).

-Steve

by **webdev** » Fri Jul 24, 2009 1:58 am

shannah wrote:In your language file (e.g. lang/en.ini) you would add a line with:
Code: Select all
import_filters:filter_name = "This is a description of this import filter".

okay got it but the correct message key is "import_filters:filter_name:help"

shannah wrote:If you don't already have a language file, you just create a directory named 'lang' in your application directory. Then add a file named en.ini (english language file). You could do a french translation by adding fr.ini, etc....

i think that your application skeleton should be more populated in order to show the user what additional config files are available. Don't bother putting some empty files, as long as it doesn't harm your application, at least it's clear out of the box which files can be customized and where.

shannah wrote:You can return a PEAR_Error (i.e. return PEAR::raiseError("This is the error"). But it may come out a bit ugly. And it wouldn't allow the import to go through smoothly.

You could store a message in the session --msg variable to be displayed on the preview page, if you just wanted to display a warning.

e.g.
$_SESSION['--msg'] = "3 records could not be imported because they contain profanity";

I haven't tried the PEAR way and as you say it may not be proper i'll leave it for now.
Now using $_SESSION['--msg'] in my import filter produces some weird results. First if i simply use it like that i get an empty orange message div at the top. Then browsing your code i saw that $_SESSION['msg'] was used as well so i tired to use it instead but i just got nothing then using them both shows the message properly in the orange div.

i use the following code :

Code: Select all: $_SESSION['msg'] .= "3 records could not be imported because they contain profanity"; $_SESSION['--msg'] .= $_SESSION['msg'];

There are two other things i'd like to fix now in the import process.

i'd like to get rid of the header message during all the import steps, my message is as follows :

Found 3 or 3 records in table objet [DROPDOWN BOX]
Now Showing 1 of 3
Current Record: 08/01/01/01/21

Then i'd like to customize the recordgrid showing the results, width is too narrow and one cell having a long text should cover most of it.

Anything you can point me to to help me do that ?

And also i'd like to get my default import filter directly selected as i only have one, it doesn't make much sense making the user select it, which he will forget inevitably.
Finally i'll probably try to remove the "default field value form" at the bottom as i won't have any use for it and it just complicates things for the the average user, who will think twice about anything he sees, ending up not knowing what to do and asking me for clues..

Sorry for all the questions, i'll probably work my way thru some of them i just don't have much time to understand all your quite sophisticated php app.

thx again.

by **webdev** » Fri Jul 24, 2009 3:11 am

quick follow-up on the single selected import filter, i was able to do that by modifying those few lines :

in ImportForm.php, near line 215, replace :

Code: Select all: $options = array(0=>df_translate('scripts.GLOBAL.FORMS.OPTION_PLEASE_SELECT','Please select ...'));

with

Code: Select all: if ( count($filters) != 1) $options = array(0=>df_translate('scripts.GLOBAL.FORMS.OPTION_PLEASE_SELECT','Please select ...')); else $options = array();

it simply dismisses the "Select filter" entry thus giving the only filter the "selected" position.

then in order to properly get the help message displayed right away, in the template Dataface_import_RelatedRecords.html, near line 70 :
replace the {literal} block :

Code: Select all: {literal} function updateFilterDescription(filterName){ ... ... //--></script> {/literal}

with

Code: Select all: {literal} function updateFilterDescription(filterName){ var filterDiv = document.getElementById('import-filter-description'); filterDiv.innerHTML = filterDiv.__filter_descriptions[filterName]; } {/literal} {if $filters|@count == 1 } {foreach from=$filters item=filterName} filterDiv.innerHTML = filterDiv.__filter_descriptions['{$filterName|addslashes}']; {/foreach} {/if} {literal} //--></script> {/literal}

it's not very nice but i couldn't find a way to access filterDiv.__filter_descriptions first element in javascript and don't know if we can get the $filters first element directly in the template language either, so i used the enumeration loop..
Can probably be done more elegantly..

anyways, it works.

one thing could be done to set the default filter name, in the actions.ini maybe, but i'm not sure how that works.
in the same way we could set a :

Code: Select all: imports_filter:<filtername>:usedefaultvalues = [0/1]

in order to specify if that import filter should get the "default values" form at the bottom..

by **webdev** » Mon Jul 27, 2009 2:45 am

another follow-up on my setup, import filter part :

first if you want to "cleanup" the import page here's what i did, it removes the "current record info" header from the page :

- make a copy of "templates/Dataface_Record_Template.html" and rename it "templates/Dataface_Import_Template.html"

in that new file remove the following lines :

Code: Select all: {if $ENV.prefs.show_result_controller} {block name="before_details_controller"} <div>{include file="Dataface_Details_Controller.html"}</div> {block name="after_details_controller"} {/if} {block name="before_record_heading"} {define_slot name="record_heading"} <h2>{if $ENV.record}<span><b>{translate id="templates.Dataface_Record_Template.HEADING_CURRENT_RECORD"}Current Record{/translate}:</b> </span>{$ENV.record->getTitle()}{else}{translate id="scripts.GLOBAL.NO_RECORDS_MATCHED_REQUEST"}No Records Matched your Request{/translate}{/if}</h2> {/define_slot} {block name="after_record_heading"}

the header is gone and you can further customize your import page as it no longer depends on the "Record_Template".

- in "templates/Dataface_Import_RelatedRecords.html", edit the one line :
from

Code: Select all: {use_macro file="Dataface_Record_Template.html"}

to

Code: Select all: {use_macro file="Dataface_Import_Template.html"}

Additionally if you want to get rid of the "Default values" bottom part of the form you can comment the following lines in "Dataface/ImportForm.php" :

Code: Select all: /* $defaultValsEl =& $this->addElement( 'optionalelement', '__default_values__', 'Default Values'); require_once 'dataface-public-api.php'; foreach (array_keys($fields) as $field){ if ( $fields[$field]['widget']['type'] == 'hidden' ){ $fields[$field]['widget']['type'] = 'text'; } $tempEl =& $df_factory->_buildWidget($fields[$field]); if (!$tempEl->getLabel() || $tempEl->_type == 'hidden' ) { } else{ $defaultValsEl->addField($tempEl); } unset($tempEl); } */

Unfortunately that disables the "Default Values" for all your import filters but if you need a very basic import page for inexperienced users then it's probably better to remove that extra feature from their sight..

Now concerning the import process i also noticed the previous bug i mentioned in a previous post http://xataface.com/forum/viewtopic.php?p=23963#23963 (when no data is actually returned by the filter script) but here when submitting the import with the step2 and NONE of the records are actually imported due to database constraints.

For example if i try to import the same list of records two times and one of the fields in my table has a UNIQUE constraint, then the duplicate records are "rejected" during the "db insert" process, as they should be, and so probably not reported as "inserted" to the import script which then for some reason redirects to the step2 (recordgrid view) import page and reports that the "import was successful".
Of course there are no records available in that "new" import step so we see the same errors as described before.

I'll update the dug report with that if it's any help.
In that case it maybe be just a redirect problem, we should simply be redirected to the list view as with truly successful imports where the "import successfull" message should be shown.

That leads me to another comment that error messages from the database should be mentioned somewhere to the user and the "success message" should show a count of the successfully inserted records with some kind of reminder of the operation like, "Successfully inserted 10 records from your import list of 10 records".

new user questions and smarty bug

new user questions and smarty bug

Who is online