Virtual tables

Q:

We want to use sql server to act as our database. From what I have read, it sounds like vTables should be able to do this.

Lianja connects to the DSN I have setup without problem. Then I created a new database and imported the tables from the DSN and created vtables for all. Just to test things out, we created a grid using the vtable vt_client. The data seems to display without any issue.

Problem: 
The data displays fine and editing the data or adding to it seems to work as well. We can make changes and add new entries and it saves it successfully. BUT the data changes doesn’t communicate through to sql server. How do we accomplish this?

A:

Make sure that you have specified a primarykey for the Virtual Table. This is the KEYFIELDLIST clause in the CREATE VIRTUALTABLE command or ‘4 Key field list’ in the MODIFY A VIRTUAL TABLE dialog. This can also be used to exclude fields from the internal WHERE clause used for updates and deletes, e.g. ‘*,-id,-last_update’ would use all fields except the id and last_update fields.

Also use the updatefieldlist PROPERTY to exclude any fields that should not be updated, e.g. ‘updatefieldlist=*,-id,-last_update’.

The dbtype property can also be specified: ‘dbtype=mssql’.

A2:

You need to make sure that you have a primary key on your SQL Server table and that you specify it as the primary key in you Virtual table definition.


dynamic connectstring.

Code:
use mydb.dbo.mytable connstr "driver={SQL Server};server=myserver;Trusted_Connection=Yes"

//Also:

use mydb.dbo.mytable connstr "driver={SQL server};server=myserver;Trusted_Connection=Yes" alias mytemp.

select mytemp
browse

 


When creating virtual tables that use one of the newer versions of SQL Server, be sure to use the latest available ODBC drivers.

Code:
create virtualtable vt_mytable connstr 'driver={={ODBC Driver 13 for SQL Server};server=.;uid=xxxx;pwd=xxxx' keyfield 'PK1' properties 'keepalive=1;dbtype=mssql' as select * from mytable.

The performance improvement is very noticeable.


Q:

I’ve set up a grid with a virtual table (using VFP ODBC.) I set up a calculated field (balance = totalbudget-totalexpenses.) It works fine and calculate accurately in development view, but not so in app view, web view, tablet view, etc. There it gives me a zero.

Name:  lianja grid problems.jpg Views: 31 Size:  76.7 KB

A:

Look at the example web apps and verify that the server is running and they are working.

You need to make sure you have deployed your database containing your virtual table definitions.

Calculated columns are not yet available in the web/mobile grids.

Are you hiding the section header? [ I’m hiding a few columns, not just headers (as far as I can tell.) ] If you look at the examples you will note that this works as expected so sonething else is causing this.


Q:

My app works fine in development and web app view, but when I use Preview to see it running in the browser, I get a Server connection lost -message. Demo apps run fine, as do test apps using Lianja SQL server. I made sure that the virtual table has a primary key. Is ODBC problematic with apps running in the browser, or am I maybe missing something?

I will try the demo that use ODBC/VT’s once I have installed MySQL on my dev machine to see if I missed something in the setup.

Name:  Lianja Server Failed to Respond.jpg Views: 76 Size:  26.9 KB

A:

No, ODBC is fine in web and mobile apps.

Have you deployed the database containing your virtual table? Web app view uses your development data whereas preview uses deployed data so the database needs to be deployed.


Q:

Are there any restrictions with regards to accessing virtual tables using a SQL SELECT statement?

I’m finding that when working in the Console work space:

with the southwind database open…

select * from vt_orders –> expected 822 records.
select * from southwind!vt_orders –> 0 records with no error.

with the southwind database closed…

select * from vt_orders –> file does not exist error.
select * from southwind!vt_orders –> file does not exist error.

 

A:

You need to create vt_orders2 as select * from southwind!orders and it works with and without the database open.

with the southwind database open…
select * from vt_orders2 –> expected 822 records.
select * from southwind!vt_orders2 –> expected 822 records. 

with the southwind database closed…
select * from vt_orders2 –> file does not exist error as this virtualtable is in a database not a free table
select * from southwind!vt_orders2 –> expected 822 records. 

Name:  Screen Shot 2017-04-08 at 3.03.06 PM.jpg Views: 25 Size:  135.4 KB

Name:  Screen Shot 2017-04-08 at 3.09.39 PM.jpg Views: 25 Size:  139.4 KB

The basis behind using virtual tables with OData is that you create the VT with the join(s) you require.

You can however create VTs which are stored procedure calls.

Currently these stored procedures are executed on the ODBC target backend but there are plans to add and recognize “localcall” which will provide the ability to call local Lianja/VFP stored procedures to fulfill a request. The way this handles parameters has not yet been finalized but its likely to be with a $params argument extension to OData.

This also means that if you want the “any data” back end capabilities that virtual tables provide, and you plan on accessing them via the OData interface, you won’t be able to use any complex joins unless you perform the join on the client or in a server side function.”

Thats not strictly speaking true.

If you want “any backend” you create your VT’s in separate database containers and assign these to users in their tenancy.

I think “localcall” is the best way forward for joining disparate data sources. I will look into it.




You can access remote databases readonly without the need to setup a VirtualTable definition.

Code:
use sales.customers connstr "lianja_mssqltest"
list first 25

also…

Code:
use sales.customers connstr "lianja_mssqltest" where condition
list first 25

and…

Code:
use sales.customers connstr "lianja_mssqltest" as select * from sales.customers where condition order by column
list first 25

Once you have these “local cursors” you can join disparate databases as Lianja will build the required join indexes dynamically for you.

So, you can dynamically fetch data from remote databases with very little effort. If you need to update data you will need to setup a VirtualTable definition that describes the primary key and other pertinent information such as updatefldlist etc.

Here is an example.

Name:  Screen Shot 2017-04-08 at 3.40.26 PM.jpg Views: 66 Size:  140.9 KB


Q2:

It also works when specifying a dynamic connectstring. Like so

use mydb.dbo.mytable connstr “driver={SQL server};server=myserver;Trusted_Connection=Yes”

2 questions.
1. once I call the command, how can I reference the cursor? I can list the data, but I can’t seem to browse it.
2. When I call the command a second time, it is telling me the database is already open. Is there a connection that I need to close?

A:

Give it an alias name then SELECT aliasname

LIST Status to see what the default was.

One you have performed the query against MSSQL you can get a reference to the CursorAdaptor() and perform requery() on it.

use mydb.dbo.mytable connstr “driver={SQL server};server=myserver;Trusted_Connection=Yes” alias mytemp.

select mytemp
browse


Q:

I setup an ODBC connection to my sql server, and am now working with vTables.

So on the table Clients, my name field type in sql server is nvarchar(Max). I am trying to make a Lianja table that displays all my clients, but in the name field I only get the word Memo, that looks like a hyperlink.

I have seen in a form I can get it to display correctly if I use a textbox. But how do I manage this in a table?

A:

A varchar maps into a memo field as it can be an indeterminate length. I’m puzzled as to why a name field would be a varchar as it is problematic to work with as both a primary key and in IDEs.

Nevertheless you could use data mapping to convert the varchar to a fixed length and pad it out if nescessary.


Q:

The data i.e. *.dbf files (VFP free tables) are in one drive (D, different from the application drive (C. These tables are updated daily by the data entry staff.

Besides using virtual tables, is there another way to develop a Web App to query/update the records ?

A:

why you do not want to use VirtualTable?

it’s simple and fast..


 VT can actually run a stored procedure or make a web service call or odata etc., and the return the resultset.

Advertisements

VFP ODBC drivers

Q:

Generally we write our apps to hit more than one backend.

Various of our routines (e.g., “what is the current database”) have to know the backend type in order to use the appropriate command to return the database name.

To do that, we need to know the connection type before we have created a cursor (if we have created one, the cursoradapter dbtype property does the trick). Note that from the application’s perspective, it was given a connection string and create a connection with it.

To summarize:

1) how to use a handle to find the dbtype in the absence of a cursor?

2) with regard to Lianja ODBC, how to find the current database? Essentially, return database() from the Lianja SQL Server. Note that if the Catalog Column from systables is always the name of the database, this one is solved. If not, then I need another way. IOW: is there a 1:1 correspondence between the database directory name and the .cat file name?

A:

There is no way with ODBC to determine the type of data that you are connected to.

Lianja uses heuristics to do this internally and then sets dbtype as you mentioned.

There is also no way to determine the database that is specified in the ODBC connection string. You can however evaluate an expression and get its value e.g.

select database() from sysresultset

Yes the database directory name and the .cat file are the same.


Q:

I need to be able to do select statements from Lianja against the vfp table created by xcase so I can scan though and do the work needed to crate / update table, create indexes in Lianja etc.

Since the tables don’t belong to a datatbase what is the best way to do this? I assume I have to have database to use ODBC?

I have tried with a test table ( this one does belong to a database)

select * from c:\1temp\daddress into cursor temp1

and get an error

Fri Apr 7 18:12:12 2017
**** Lianja error ****
DADDRESS.LA10AUTHOR
^
Fatal I/O error writing record 1 to table – errno 22

Q2:

I’ve just tried to copy one of the tables to a Lianja table using
use c:\1temp\ddent noupdate
copy to c:\1temp\l_ddent with production

and get

Fri Apr 7 21:10:46 2017
**** Lianja error ****
copy to c:\1temp\l_ddent with production
^
ERROR – invalid record length

Hank tells me that the Xcase case tables are in foxpro 2.x rather than VFP type so maybe that is an issue.

A:

I am able to USE that table and COPY TO another_name to put it in Lianja format.

SQL SELECT is working Ok in 3,.4 with that table too although its not really supported as you need to convert to Lianja format so that the query optimizer can do its work.

Name:  Screen Shot 2017-04-08 at 2.27.43 PM.jpg Views: 27 Size:  89.1 KB

Just USE tablename ALIAS whatever then perform your SQL SELECTSs.

Just an FYI, Lianja supports the following XBase file formats natively readonly.

VFP
FoxPro 2.x
Clipper

A2:

in my opinion, the best thing is to create an ODBC link to your xcase table, then create a VT on that table.

With a old FoxPro table, I do that.


Q:

Can you make ODBC links to tables that are not part of a database?

The code I have looked at for the ODBC connection wants the name of the database

A:

For free VFP table you need VFP ODBC driver.

 

 

 

VFP migration

Q:

I have an existing project deployed in visual foxpro 5 and i was trying import project for Lianja. I downloaded linja trial for 30 days to make tests because my goal is putting the tables of project in a database so i think lianja could help me.
So i created an app in visual foxpro and imported my project for there but i can’t run the project to see the apllication. how can i do that?

A:

Information on Visual FoxPro imports and scripting can be found here: http://www.lianja.com/doc/index.php/…xPro_Scripting, including:

Importing Visual FoxPro Databases and Tables – http://www.lianja.com/doc/index.php/…ses_and_Tables
Importing Visual FoxPro Files – http://www.lianja.com/doc/index.php/…l_FoxPro_Files

Imports of Visual FoxPro projects, classes and forms extract the data from the VFP tables and generate source code scripts in your App. Start from the Lianja/VFP Command Window in the Console Workspace and run your scripts using DO or DO FORM and issue any required SET CLASSLIB or SET PROCEDURE commands to reference libraries.

Check the Error and Log windows in the Console Workspace for any issues. Your scripts can be edited in the Apps Workspace.

Form scripts (*.scp) can also be referenced from a Page (Pages Workspace -> Form Tools -> Page), in which case they are embedded in a full-Page sized VFP Custom Section in your App. To modify the Custom Section, edit the Form script on which it is based, to add other Pages to the App, use any of the wide range of Lianja in-built Sections.

After using the dbc importer, you will have a Lianja database with your tables and production indexes. You can then access these from the Data Workspace, using OPEN DATABASE, SQL statements and USE and other NoSQL commands from the Console, from scripts and programs or using drag and drop and attribute assignment in standard Lianja sections.

Note that it is recommended to rebuild your forms using the Lianja visual Page Builder and drag ‘n’ drop Lianja standard sections and combine these with your existing VFP background business logic code. This, along with being the easiest and quickest way to migrate – providing you write the client event delegates in JavaScript (with Lianja/VFP extensions) – will have the added advantage that you will also be able to run your Lianja Apps on web or mobile devices.

You will see the source code for your imported project files in the Files explorer in the Apps workspace with your App open. Source from forms will have a .scp extension, source from class libraries a .vcp extension, .prg files remain unchanged.

You can then run the scripts/programs, edit the scripts, and embed in a Page if required.


Today user expectations from applications are very different from those VFP days.
Because of the technology gap I am afraid there is no free lunch in the migration.

Lianja is “cross” on so many levels (Win/Mac/Linux , LAN/internet/Cloud, desktop/web/mobile…)
Choosing among standard sections, custom sections and canvas sections…
VFP/JavaScript/PHP/Python…
Lianja is huge. “Cross…” and “multi” are a plus, but in the same time it is a problem – because of all complexity.

Is Lianja for you – it is a tough question. You need to know where you are headed with your apps.
I suggest you read more about Lianja concept: http://www.lianja.com/community/show…Lianja-concept

VFP apps need to be broken down to smaller pieces and to be rearranged a lot for today modern technologies:
App Center (and Categories winthin) -> Apps -> Pages (and Page centers) -> Sections as visual representation of SQL tables
What about strict separation of server side computing from the client side computing in our old VFP apps?
That is why I do not believe in easy importing on button click.
Do not expect that you will automatically get GUI responsiveness and all to work in web browsers and mobile.

Starting from scratch, at least, we can use VFP syntax knowledge in delegates and progress step by step in converting our apps.
Your VFP knowledge are very valuable and useful here, unlike in some other alternatives where you need to learn completlly new syntax.

Advice? Do begin with some small part of your complex app and see how are you doing.


The (single) most important document you should read before you start developing Apps in Lianja.

Variable

Q:

I am trying to upload an image file (http post “multipart/form-data”) to a “storeimage.rsp” to process. However, it returns the following error:

Error message:
ARRAY reference is out of bounds

Error line:
_FILES[0].tmp_nam

————————————————
How can i read and store the file with the _files[]?

A:

The array starts from 1 not 0.

There is an example included in the distribution in C:\lianja\cloudserver\tenants\public\wwwroot\examp les\example_upload.html which calls C:\lianja\cloudserver\tenants\public\wwwroot\examp les\example_upload.rsp.

You can run it as http://127.0.0.1:8001/examples/example_upload.html

Note that example_upload.html references ‘http://127.0.0.1/examples/example_upload.rsp’ – if you have not enabled port 80 on the Lianja Server (or set up the ISAPI IIS extension), change this to ‘http://127.0.0.1:8001/examples/example_upload.rsp’.


AMEMBERS()  works with object() also:

Code:
lo = object()
lo.hank = "here"
lo.yvonne = "over there"
lo.barry = "way over there"
? amembers(awhowhere,lo)
? lo
? awhowhere

result:

3
Object (refcnt=1)
(
[hank] => here
[yvonne] => over there
[barry] => way over there
)
Array (refcnt=1)
(
[1] = HANK
[2] = YVONNE
[3] = BARRY
)

It also works with the SCATTER MEMO NAME <name> object.


AMEMBERS() does not work with the UI framework classes.


Q:

I’ve a form, with a new container, with a button.

On the “cick” event, I call a procedure “sp_close_handler”

Code:
oContBottom.addObject("sp_close", "commandbutton")
sp_close.move(1, 1, 60, 20)
sp_close.text = "Close"
sp_close.click = sp_close_handler
Code:
proc sp_close_handler()
	p_myform.close()
endproc

can I send a parameter on the call?
for example:

Code:
oContBottom.addObject("sp_close", "commandbutton")
sp_close.move(1, 1, 60, 20)
sp_close.text = "Close"
sp_close.click = sp_close_handler with "aaa"

 

Code:
proc sp_close_handler(m_text)
	lianja.showmessage(m_text)
	p_myform.close()
endproc

naturally, this code not work.. 🙂

A:

use use a public variable to pass the text.

If the event to which you you assign sp_close_handler has one or more params, they should be passed to the routine you assign.

However, the click() event does not have params.

You could use a public var for this. In Lianja, where we aren’t creating large monolithic applications, the usual best practice of “don’t use public vars” doesn’t really apply. When I have a lot of them, I do place them on an object() when in VFP, or in a dynamic JavaScript array

VFP

Code:
goMyVars = object()
goMyVars.Phabio = "lasagna lover"

JavaScript

Code:
goMyVars = {};
goMyVars.Phabio = "lasagna lover";

 

Validation

Q:

I’m tryng to use the classic “Browse” command
I I wryte:

Code:
open database southwind
use example
browse field title:c="MR.Mrs,Miss,Mr",;
last_name:w=10:V=Validate(),;
first_name

proc validate()

endproc

in the validate, how can I test the value just insert in the “last_name” field?

if I test “last_value” I see the previous value..

A:

Pass “{}” to the validation proc.

Code:
valproc("{}")


Q:

I’m trying to do validation for a Textbox field on a canvas.

Suppose I set Validation to !empty(“{}”) and Error to “Field must not be empty”.

1. When I press the tab key, the error message appears in the yellow error dialog, but the cursor moves to the next field. Is this expected behavior? I would expect the focus to be kept on the offending field.

2. If the field is empty to begin with, tabbing through the field does not generate the error dialog. Is this expected behavior? The field fails the validation, so I would expect the error dialog to appear and focus to remain on the offending field.

3. What is the recommended approach if the validation for a field has multiple components, requiring varying error messages for the field?

4. Are there other preferred alternatives (different delegates, etc.) for doing field level validation?

A:

The Validation expression is checked on exiting the field when the field value has been modified.
This expression can also be a call to a function/procedure/prg, so you could handle your varying error messages their, e.g.

Validation: myfunc(“{}”)

Code:
//myfunc.prg
param fld_value
do case
case fld_value = "something"
...

 

You also have Section and Page level ‘Valid when’ delegates and the Section ‘Before data update’ delegate where data validation can be carried out:

Users and Roles

Lianja already supports user roles and permissions for UI elements and now in Lianja 3.4 user roles extend into the core of the database engine to provide row level security.

Row Level Security (RLS) is a concept that provides security at the row level within the database layer, instead of at the application layer. With this new feature you can implement RLS without changing your application code.

Row-Level Security enables customers to control access to rows in a database table based on the roles (or groups) of the user accessing data.

Row-Level Security (RLS) simplifies the design and coding of security in your application. RLS enables you to implement restrictions on data row access. For example ensuring that workers can access only those data rows that are pertinent to their department, or restricting a customer’s data access to only the data relevant to their company.

The access restriction logic is located in the database engine rather than away from the data in applications. The database system applies the access restrictions every time that data access is attempted.

While architecting this functionality we looked at how it is implemented in other industry standard SQL engines and found all of them to be overly complex and long winded, so we came up with a simpler approach which is more flexible.

Row level security in Lianja is accomplished using row filters. Each row filter is associated with a tenancy, database, table and role name.

When a user authenticates the tenancy and username determine the roles assigned to the user.

Whenever a database table is accessed, the database engine will lookup the row filters associated with the table based on the roles that the authenticated user has been assigned.

If a user has a specific role and a row filter exists for that named role then the row filter associated with that role is included (OR) from the records that can be read.

If the role name is negated e.g. ~manager and the user does not have manager role, then the row filter is excluded (AND NOT) from the records that can be read.

These row filters operate across native Lianja tables and virtual tables, so you can use them with MSSQL, MySQL, PostgreSQL and other third party SQL databases also.

When defining row filters for Virtual tables you should specify the name of the virtual table as the table e.g. vt_customers.

Row filters are transparent to the application in desktop, web and mobile apps as well as OData API calls. In other words no changes in your applications is required in order to benefit from row level security in Lianja.

You create row filters in the users workspace.

Name:  Screen Shot 2017-02-17 at 12.03.51 PM.jpg Views: 62 Size:  47.9 KB

Row filters are stored in the system!sysrowfilters table.

This table has the following table structure:

Structure for table : sysrowfilters.dbf
Alias : sysrowfilters
Number of records : 2
Date of creation : 02/17/2017
Date of last update : 02/17/2017
DES3 Encrypted : No
CursorAdaptor : No
Field Field Name Type Width Dec Description
1 DOMAIN Character 80 Domain
2 DATABASE Character 80 Database
3 TABLE Character 80 Table
4 ROLE Character 80 Role
5 ROWFILTER Memo 8 RowFilter
** Total ** 329

Because row filters are stored in a database table you can also use standard SQL commands to perform CRUD operations on the row filters.

insert into system!sysrowfilters values (‘public’, ‘southwind’, ‘customers’, ‘bname’, [startsWith(customerid, ‘B’)])

Notice that ‘public‘ is the tenancy that a authenticated user belongs to and ‘bname’ is the role you have assigned to the user. A user can have multiple roles.

If the role you specify is prefixed with a ~ e.g.

insert into system!sysrowfilters values (‘public’, ‘southwind’, ‘customers’, ‘~cname‘, [startsWith(customerid, ‘C’)])

and the authenticated user does not have the cname role, then all rows where the customerid starts with a C are excluded from what the user can see.

You can test data access for different users and roles using the following commands interactively in the console workspace.

open database southwind
use customers
list status
list

set rowfilter on | off
set rowfilter to <expression>

The list status command now includes the roles and row filters for each table.


Q:

in a context of a multi company DB .. I can filter data for company?

Customers Table structure:
CompanyID, customerId, customerName

if the user is assigned to the comapany “A”
i can:
insert into system! sysrowfilters values ( ‘public’, ‘Southwind’, ‘customers’, ‘~ CompanyID’ [= ‘A’])

or
update …

And the user see and edit only the Customers records where “CompanyId” = “A”?

In this way I can create a multi tenancy app in a easy way..

A:

Lianja already provides you with multi tenancy databases. You could use row filters but having a separate database for each company/tenant is a better choice in my opinion.

Row filters are better suited to providing views on data depending on user roles.

In your example code you would set the ROLE to ‘companya’ and the ROWFILTER to companyid=’A’

Then you assign the companya role to users in that company.


Q:

Suppose we have many contract staff who join and go on different date. When I add these new users, how can I auto set it to an “expired role” (i.e. deny further login beyond expiry will prompt “Contract expired.”) say 1 year or 1/2 year from the date of entry?

A:

The users table does not have an expiry date column. You could however add a role e.g. expires20170701 and have a background task running once a day that removes users who have expired. This could be a .rsp page.

A2:

I have added an “Expires” column into system!sysroles table in Lianja 3.5. If a user attempt to login and an expiry date has been specified then the login is rejected if it has matured.


 

UI State

Lianja States provide you with a powerful mechanism for handling complex business processes using a “State machine” which is built into Lianja.

State machines have many interesting uses. States can easily describe user interfaces but they can also be used to model high-level business processes and many other things.

In a Lianja App, each UI element (App, Pages, Sections and FormItems) contains a (optional) comma separated list of “States” that the UI element responds to. In Lianja 3.2 we have added a new delegate called stateChanged(cState) which is called whenever a UI State is changed. This provides you with a programmable hook for managing App state.

e.g:

Lianja.showDocument(“changestate:namelist”)
or
Lianja.showDocument(“page: page1?action=changestate&state=namelist”)
or
Lianja.showDocument(“section:section1?action=chang estate&state=namelist”)

The delegate is called with one argument, the name of the state. Note that the namelist is a comma separated list so you can trigger multiple UI state changes at once.

When a “resetstate:namelist” is applied to the App the delegate is called with a single argument which is preceeded by a “-” sign e.g. “-editing”.

Note that changing UI state is only handled in runtime mode not the development view.