Tuesday, May 21, 2013

SQL Server Compact Code Snippet of the Week #15 : flush data to disk immediately

Under normal operation, SQL Server Compact keeps all pending disk writes in memory, and flushes them to disk at least every 10 seconds. The connection string property to control this is called “Flush Interval”, and valid values are between 1 and 1000 seconds. But you may want to flush to disk immediately under certain circumstances, and this weeks code snippet demonstrates how to do just that. This is possible via the CommitMode property on the SqlCeTransaction object Commit method, as demonstrated below:

using (SqlCeConnection conn = new SqlCeConnection(@"Data Source=C:\data\AdventureWorks.sdf;"))
{
    conn.Open();
    // Start a local transaction; SQL Server Compact supports the following
    // isolation levels: ReadCommitted, RepeatableRead, Serializable
    using (SqlCeTransaction tx = conn.BeginTransaction(IsolationLevel.ReadCommitted))
    {
        using (SqlCeCommand cmd1 = conn.CreateCommand())
        {
            // To enlist a command in a transaction, set the Transaction property
            cmd1.Transaction = tx;
            try
            {
                cmd1.CommandText = "INSERT INTO FactSalesQuota " +
                    "(EmployeeKey, TimeKey, SalesAmountQuota) " +
                    "VALUES (2, 1158, 150000.00)";
                cmd1.ExecuteNonQuery();

                // Commit the changes to disk immediately, if everything above succeeded;
                // Use Deferred mode for optimal performance; the changes will
                // be flashed to disk within the timespan specified in the
                // ConnectionString 'FLUSH INTERVAL' property (default 10 seconds);
                //
                tx.Commit(CommitMode.Immediate);
            }

            catch (Exception)
            {
                tx.Rollback();
            }
        }
    }
}

 

Thursday, May 9, 2013

SQL Server Compact Code Snippet of the Week #14 : script all data in a table

Another entry in the scripting API samples, you will find an overview of getting started with the API here.

This weeks entry demonstrates the general pattern for scripting smaller chunks of SQL based on a single table, in this case a INSERT statement for each row in the table.

using (IRepository repository = new DBRepository(@"Data Source=C:\Northwind.sdf"))
{
    Generator generator = new Generator(repository, null);
    foreach (var tableName in repository.GetAllTableNames())
    {
        generator.GenerateTableContent(tableName, false);
    }
    System.IO.File.WriteAllText(@"C:\script.sqlce", generator.GeneratedScript);
}


For SQL Server Compact 4.0, the scripting library is now available on NuGet, making it even easier to get started. https://nuget.org/packages/ErikEJ.SqlCeScripting

Friday, May 3, 2013

SQL Server Compact Code Snippet of the Week #13 : reseed (reset) an IDENTITY column

A question that keeps re-appearing in the forum is, how can I reseed/reset an IDENTITY column in SQL Server Compact, as no DBCC command is available. You can simply use a special syntax of the ALTER TABLE command to do this, as follows:

ALTER TABLE [MyTableName] ALTER COLUMN [Id] IDENTITY (1, 1)

This will make the value of the next generated IDENTITY value 1 and increment with 1. Notice that you do not specify the column type, only the IDENTITY specification.

Wednesday, April 17, 2013

SQL Server Compact Code Snippet of the Week #12 : get the SQL Server Compact runtime version

Your app/web site may require a specific build version of the SQL Server Compact runtime, due to dependency on a bug fix, for example. This weeks code snippet will demonstrate how to get that information. I have an older blog post here, that goes into deep details about the various ways to get SQL Server Compact related version information.

To get the SQL Server Compact build version for 3.5 SP2 and 4.0, you can simply use:

var ver = new System.Data.SqlServerCe.SqlCeConnection().ServerVersion;

MSDN docs here.

For SQL Server Compact 4.0 SP1, this returns: 4.0.8876.1.

And for SQL Server Compact 3.5 SP2 CU6, the value would be: 3.5.8088.0

Wednesday, April 10, 2013

Generate a Windows Phone 8 Local Database DataContext from an existing database

Despite my tooling for this having been available for more than 18 months, I have never posted a dedicated blog post for this subject. This post intends to remedy this.

The Local Database (a SQL Server Compact database accessed via LINQ to SQL) is a data access API available on Windows Phone 7.5 and 8.

The Microsoft documentation (listed here on my blog, together with many other useful Windows Phone Local Database links) always describes a Code First workflow, which makes it cumbersome to reuse existing effort in creating a SQL Server Compact database schema for Windows Mobile or desktop, and  also makes it hard to distribute a database prepopulated with data together with your app. My tooling, which is available with the SQL Server Toolbox Visual Studio add-in, and also available in a simpler form with the new standalone edition of the Toolbox for SQL Server Compact 3.5 (currently in alpha), enables both scenarios. The standalone edition is useful for VS Express users and when you do not wish to install Visual Studio on a PC (it is a single .exe file, so very simple to distribute)

In the following walkthrough, using Visual Studio 2012, I will demonstrate how to use the SQL Server Compact Toolbox to take an existing SQL Server database and use it as an included Windows Phone database in an new (or existing) Windows Phone 8 App. The process to do this requires these steps:

- Create the SQL Server Compact database from the server database and add it to the Windows Phone project
- Generate the LINQ to SQL DataContext and releated classes.
- Use the database from code

I assume you have Visual Studio 2012 Pro or higher with the Windows Phone 8 SDK installed.

Create the SQL Server Compact database

I have created a new Windows Phone Databound App for this sample, and selected Windows Phone OS 8.0 as the target OS.

image

I then use the Toolbox to create a new SQL Server Compact 3.5 database in the folder where the Phone project resides, (you can determine the folder from by using the “Open Folder in File Explorer” context menu item).

image

I then click Create, navigate to the project folder, and type PostCodes.sdf, press OK.

image

Click OK, and a new, empty database will be added to the database list in the Toolbox:

image

Now we need to connect to the SQL Server database, and script it, then run the script against the new, empty database.

image

Create and save the database script using the Toolbox menu item above, and then open the SQL Editor against the PostCodes.sdf database file:

image

Use the Open button in the editor to load the script, and the press the Execute button to run the script.

image

Now the database contains a PostCode table (the script is available here), which has all Danish postcodes.

The final step is adding the database file to the Phone project. In Solution Explorer, select “Show all files”, and include PostCodes.sdf. In this sample scenario, we would like the database to become writable on the Phone, so include it a “Embedded Resource” – it could also be included as Content, if it was just a read-only database, read more here.

image

 

Generate the LINQ to SQL DataContext

In order to generate the DataContext based on the database, right click it in the Toolbox, and select “Add Windows Phone DataContext to current project”.
If this menu item is disabled, verify that the database file is in 3.5 format, and that the SQL Server Compact 3.5 SP2 runtime is properly installed, you can check this via the About dialog. “Yes” is required in both places, if that is not the case, repair/re-install.

image

image

image

Let’s walk through the various options on this dialog:

Context name: The name of the generated DataContext class

Namespace: Allows you to specify another namespace for the generated code

Language: You can generate C# or VB code.

Pluralize: If checked, will rename tables (Person => People) etc.

Create a file per table: Normally, just a single file is created

Advanced options:

Add schema version table: If you would like to include the database file a a writeable file, and allow use of the DatabaseSchemaUpdater class in a future app version select this option .

Add rowversion column to all tables: Checking this will ensure that all tables have a rowversion column (formerly timestamp), which enhances performance when doing UPDATE and DELETE (see my blog posts here and here)

Include ConnectionStringBuilder: Will add a LocalDatabaseConnectionStringBuilder class to the project, to help with building connection strings in a strongly typed fashion.

For this sample project, just click OK, and a PostCodesContext.cs file will be added to the project, and we are done.

image

Use the database from code

Finally, to demonstrate that we are able to include data with the app, alter the DataTemplate in MainPage.xaml as follows:

<DataTemplate>
  <StackPanel Margin="0,0,0,17">
      <TextBlock Text="{Binding Zip}" TextWrapping="Wrap" Style="{StaticResource PhoneTextExtraLargeStyle}"/>
      <TextBlock Text="{Binding City}" TextWrapping="Wrap" Margin="12,-6,12,0" Style="{StaticResource PhoneTextSubtleStyle}"/>
  </StackPanel>
</DataTemplate>

Replace the OnNavigatedTo event handler in MainPage.xaml.cs with the following code:

        protected override void OnNavigatedTo(NavigationEventArgs e)
{
using (PostCodesContext ctx = new PostCodesContext(PostCodesContext.ConnectionString))
{
ctx.CreateIfNotExists();
ctx.LogDebug = true;
MainLongListSelector.ItemsSource = ctx.PostCode.ToList();
}
}



This code initialize a new PostCodesContext instance (embraced in “using”, as it is Disposable). The CreateIfNotExists method extracts the PostCodes.sdf embedded resource from the project, and copies it to isolated storage (feel free to look at the code). Setting LogDebug to true will show all SQL statements as text in the Debug window while debugging:


SELECT [t0].[Id], [t0].[Zip], [t0].[City], [t0].[Street], [t0].[Company], [t0].[IsProvince], [t0].[rowguid] AS [Rowguid], [t0].[ts] AS [Ts]
FROM [PostCode] AS [t0]


And finally, calling ToList() will execute the SELECT and return a list of PostCode objects, that is the bound to the ItemsSource property of the LongListSelector.


Result:


pc


Summary


Let us finish with a summary of advantages of this approach:
- Use desktop database tools for data population and schema design
- Saves time doing 1:1 mapping between database tables and DataContext classes
- DataContext class and entity classes are partial and can be extended
- The generated DataContext contains Index definitions (which SqlMetal does not support, as this is a Windows Phone extension)
- The generated DataContext contains the CreateIfNotExists method, that optionally extracts an included database (prepopulated with data) to Isolated Storage
- The generated DataContext includes the LogDebug property, that allows you to see all SQL statements generated on the device in the debug window
- Optionally split the generated Data Context classes into multiple files
- Optionally add a Version table if you include the table with your app, and want to enable use of the schema updater functionality.
- Optionally add rowversion columns to improve UPDATE and DELETE performance
- Optionally include a ConnectionStringBuilder class to build a valid connection string in a strongly typed way,  using advanced connection string options (see some of my Phone blog posts for candidates)


Hope you find it useful.

Monday, April 8, 2013

SQL Server Compact Code Snippet of the Week #11 : detect if SQL Server Compact is available

It can sometimes by useful to find out if SQL Server Compact is available on a system, for example in installers and if your application supports several database products. This week’s code snippet allows you to detect if the ADO.NET Provider is available and properly installed (it does not detect if the unmanaged DLL files are present). The code detects both if the AD.NET Provider is present, and if the DbProvider interface is properly registered (either in machine.config or in a local config file)

public bool IsV40Installed()
{
try
{
System.Reflection.Assembly.Load("System.Data.SqlServerCe, Version=4.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91");
}
catch (System.IO.FileNotFoundException)
{
return false;
}
try
{
var factory = System.Data.Common.DbProviderFactories.GetFactory("System.Data.SqlServerCe.4.0");
}
catch (System.Configuration.ConfigurationException)
{
return false;
}
catch (System.ArgumentException)
{
return false;
}
return true;
}


Replace Version=4.0.0.0 with Version=3.5.1.0 and use System.Data.SqlServerCe.3.5 for the DbProvider invariant name for SQL Server Compact 3.5

Tuesday, March 19, 2013

WCF Data Services with Windows Phone - bandwidth requirements measured

In connection with testing Synchronization between a SQL Server Compact database on Windows Phone 8 and a SQL Server database (via a WCF Data Services service hosted in IIS), I have done some interesting observations regarding bandwidth requirements, that I would like to share.

I am testing against the Chinook sample database, by downloading the entire PlaylistTrack table (8715 rows) to the device via my WCF Data Services service. On the server side, I am using the latest release version of the WCF Data Services server components, version 5.3.0. Version 5.1 or later includes the newer lightweight JSON format (just to compare I am also including sizes for the previous JSON format)

On the server side, I have created a ASP.NET Web Application with a WCF Data Service, that exposes the Chinook database on my SQL Server via an Entity Framework DbContext. The power of WCF Data Services is that this requires basically no code to configure. I have configured my service like this:

    public class SyncService : DataService<ChinookEntities>
{
// This method is called only once to initialize service-wide policies.
public static void InitializeService(DataServiceConfiguration config)
{
config.UseVerboseErrors = true;
//config.SetEntitySetAccessRule("TrackPurchases", EntitySetRights.WriteAppend);
config.SetEntitySetAccessRule("*", EntitySetRights.AllRead);
config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V3;
}
}




In order to access the IIS Express hosted service from my Windows Phone 8 emulator, I followed the instructions here: http://msdn.microsoft.com/en-us/library/windowsphone/develop/jj684580(v=vs.105).aspx 


To measure the size of the payload, I am using Fiddler2, by following the instructions here: http://blogs.msdn.com/b/fiddler/archive/2010/10/15/fiddler-and-the-windows-phone-emulator.aspx


The WCF Data Services team also supply a WCF Data Services client for Windows Phone, that can take advantage of a Service Reference, but this client has some severe limitations, that affects bandwidth consumption in a bad way: It only supports the XML based ATOM format, but you can enable compression, as described here: http://blogs.msdn.com/b/astoriateam/archive/2011/10/04/odata-compression-in-windows-phone-7-5-mango.aspx 


On the client side, I am simply using HttpWebRequest to call the REST url, and including support for gzip via the ICSharpCode.SharpZipLib library (for example http://nuget.org/packages/SharpZipLib-WP7/ )


Here is the implementation of the WebClient:

        static public async Task<T> GetData<T>(Uri uri, bool useJson = true, bool version3 = true, bool compress = true)
{
//uri = new Uri(uri.AbsoluteUri + "&format=json");
HttpWebRequest client = WebRequest.CreateHttp(uri);
{
if (compress)
client.Headers[HttpRequestHeader.AcceptEncoding] = "deflate, gzip";
if (version3)
{
client.Headers["MaxDataServiceVersion"] = "3.0";
}
else
{
client.Headers["MaxDataServiceVersion"] = "2.0";
}
if (useJson)
client.Accept = "application/json";

using (WebResponse response = await client.GetResponseAsync())
{
string result = await response.GetResponseText();

DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(T));
T resultType;
using (MemoryStream stream = new MemoryStream(Encoding.Unicode.GetBytes(result)))
{
resultType = (T)serializer.ReadObject(stream);
}
return resultType;
}
}
}

public static async Task<string> GetResponseText(this WebResponse response)
{
using (
Stream stream = response.IsCompressed()
? new GZipInputStream(response.GetResponseStream())
: response.GetResponseStream())
{
using (var reader = new StreamReader(stream))
{
return await reader.ReadToEndAsync();
}
}
}

public static bool IsCompressed(this WebResponse response)
{
return Regex.IsMatch((response.Headers["Content-Encoding"] ?? "")
.ToLower(), "(gzip|deflate)");
}




(I am using Microsoft.Threading.Tasks.Extensions.dll to implement GetResponseAsync)


I am using DataContext classes generated by my SQL Server Compact Toolbox for deserialization, with a small addition - I have added this attribute to all EntitySet<T> and EntityRef<T> properties (this will be included in the next Toolbox release):


[global::System.Runtime.Serialization.IgnoreDataMember]


I am calling the following URL: http://<MyIP>:2065/SyncService.svc/PlaylistTracks


This is my test code:

//ATOM-XML
await WebClient.GetData<PlaylistTrackRoot>(uri, false, false, false);
//Verbose json
await WebClient.GetData<PlaylistTrackRoot>(uri, true, false, false);
//Verbose json + gzip
await WebClient.GetData<PlaylistTrackRoot>(uri, true, false, true);
//Plain json
await WebClient.GetData<PlaylistTrackRoot>(uri, true, true, false);
//Plain json + gzip
await WebClient.GetData<PlaylistTrackRoot>(uri, true, true, true);

public class PlaylistTrackRoot { public List<PlaylistTrack> value { get; set; } }

And finally the unbelievable numbers for the download of the entire PlaylistTrack table with 8715 rows (remember, that ATOM is the default WCF Data Services client format)



























Payload typeBody size (bytes)Body size (MB)
ATOM-XML 9,322,665 (100 % – default DS client implementation)8.89 MB
JSON (verbose)5,016,977 (54 %)4.78 MB
JSON (verbose) + gzip328,410 (3,5 %)0.31 MB
JSON (plain)790,845 (8,5 %)0.75 MB
JSON (plain) + gzip43,023 (0,5 %)0.04 MB

So before you decide to use the WCF Data Services Windows Phone client, beware that the only format currently available is ATOM-XML. With the 5.1.0 or later desktop client, however, you can use the of the DataServiceContext to request JSON - ctx.Format.UseJson() – the default is still ATOM-XML.