Category Archives: Realm

Supported Linq query expressions in Realm

Sooner or later, you’ll come to the point, that you don’t only want to get objects by Id or the whole set of objects of a specific type in Realm. The easiest, but probably most unsophisticated way to do it, would be just use a function on a Linq-Where statement. That’s easy and has reasonable performance, no need to iterate and implement it yourself. But it actually is a big waste of resources, because depending upon the function, every single element needs to be evaluated.

But there is a better way to do so, eventhough it’s support in Realm is very confined. You can use Linq-Expressions. In the next table you find a plan, on what is supported or not:

ExpressionExampleRealmInfo
Int-Query
Double-Query
String-Query
class.Number==4
class.Number>3.4
class.Number<=2
class.Text==”Hello World”
String-Functions:
Contains, StartsWith, EndsWith
class.Text.StartsWith(“T”)as StringComparison you can use Ordinal or OrdinalIgnoreCase
advanced calculationclass.Number*20>100no left-side operations are supported, but right-side operations
Same-Object Comparisonclass.Number>class.Number2only comparison with constant values are supported
List-Functionsclass.List.Contains(3)
Sub-Object-Accessclass.SubClass.Number==1
Take()Take(20)
Skip()Skip(5)

That’s really not a lot. Actually it ends up with numeric type comparisons (==,<,>..) and some string-Comparer (==,Contains, StartsWith, EndsWith). I really hope Mongo Inc. will spend some more time on implementing more Linq-Expression features in Realm.

But actually there is a (little) work around (which I didn’t yet check out in terms of performance). You can use the Filter-function, which takes a query that is inspired by NSPredicate. (you can find more details here)

ExpressionFilter
advanced calculation
Same-Object ComparisonSubClass.Number>Number
List-FunctionsANY SubClassArray.Number>10
ANY or SOME works only with RealmObject-Arrays, not with atomic type arrays
Sub-Object-AccessSubClass.Number>20
Take()
Skip()

Summarizing my investigation and research on this, I can say: There’s space for improvement. I am very keen on finding out, whether MongoRealm will have a more sufficient way to query data.

User Access Management in Realm

As I already stated before, I love Realm.io. It’s a pretty cool object oriented database, with a very mighty synchronization mechanism, that helps you build mobile apps just right out of the box. But that’s not the end of the line.. It also ships with a very mighty user management feature.

When you develop mobile apps, you sooner or later want users to get authenticated. You need a user management, with registration, authentication, and permission management. That’s all but easy.

Realm delivers all of it, with some easy steps.

  1. User registration
    var usr = await User.LoginAsync(Credentials.UsernamePassword("Tom", "test", true), new Uri("realmUrl"));
    // an exception will be thrown, when the user already exists
  2. User login
    var usr = await User.LoginAsync(Credentials.UsernamePassword("Tom", "test", false), new Uri("realmUrl"));

Now that you have a user registered and logged in, you can create as many realms as you like. But you can only create realms with your UserId, in most cases this is not the Username. But don’t panic, you don’t need to store it somewhere. When you open a Realm, just add a Tilde, which will automatically get replaced with your UserId.

FullSyncConfiguration conf = new FullSyncConfiguration(new Uri("~/MyLittleRealm", UriKind.Relative));
var realm = Realm.GetInstance(conf);

Pretty easy, isn’t it? But that’s not all. Let’s assume we have an App to manage your shopping list. That’s probably enough, but you also want to share your shopping list with your wife or kids. Everyone using your App will actually have it’s own list, but can’t access the others. But you know what realms they are using, so you could synchronize them too, perhaps it is something like <UserId>/ShoppingList . But first of all you don’t have their UserId (this is not the username!) and also no permission to access it. A user needs to actively grant someone else access to their Realms:

await User.Current.ApplyPermissionsAsync(PermissionCondition.UserId("someonesUserId"), "~/ShoppingList", AccessLevel.Read | AccessLevel.Write);

You can even choose if the user can read or read and write your Realm. But how do we get the UserId?

There are two approaches to that challenge:

  1. you can store the UserId in a public realm, this is okay, but not a very secure solution
  2. your App can offer a way to directly send the UserId to others using WhatsApp, NFC or a QR-Code .. that is easy and everyone has complete control to their data (even though it’s only an ID)
  3. Didn’t I just talk about two? Ok, I think there are probably more than that, there could be something like “Sync”-Mode.. a User can write an offer in a public realm, stating he wants to share his shopping list with a user with username “xy”, the App of user “xy” could react upon that offer and publish its UserId.

By the way: When you are using an external Authentication (like an Identity-Server or similar) that offers a JWT-Token, the UserId will be the same as the Username.

Using Realm Notifications in .Net

There is a documentation on Realm.io, that describes how to hook up on a ROS (Realm Object Server) to observe changes on specific Realms, RealmObjects or whole instances. So there is no need to actually download the whole Realm, since it “sees” the changes directly on the server. It is also pretty cool, that you don’t have to load the assemblies that contain the correct RealmObjects (and version), because it uses the .Net DynamicObject feature.

https://docs.realm.io/sync/backend-integration/data-change-events#integrating-with-a-3rd-party-api

Sadly, this only works with node.js not in .Net…

There is currently only one work around, that really works with .Net Standard, that I have found. You open the specific Realm in dynamic mode and use the method SubscribeForNotifications. Be sure to have a separate Thread running for the notifications to work properly.

static async Task Main(string[] args)
{
var adminUser = await User.LoginAsync(Credentials.UsernamePassword("aUser", "****", false), new Uri("https://rosurl"));
var realmSyncConfig = new Realms.Sync.FullSyncConfiguration(
new Uri("SomeRealm", UriKind.Relative), adminUser);
realmSyncConfig.IsDynamic = true;
Nito.AsyncEx.AsyncContext.Run(async () =>
{
var realm = await Realm.GetInstanceAsync(realmSyncConfig);
var realmCollection = realm.All("Organization") as RealmCollectionBase<RealmObject>;
var token = realmCollection.SubscribeForNotifications(RealmChanged);

while (true)
{
// you can use a cancellation token here
await Task.Delay(1000);
}
token.Dispose();
realm.Dispose();
});
Console.ReadLine();
}

private static void RealmChanged(IRealmCollection<RealmObject> sender, ChangeSet changes, Exception error)
{
/* here you can handle changes */
}

There are some disadvantages when you use this solution:
1. The Realm that you want to subscribe changes on will be fully synced. (So it’s not a good solution for very large Realms)
2. You can’t just watch your whole instance, you need to actually know the full name of your Realm

Copy an entire synced Realm

In our company, we use realm.io as mobile Database with the ROS Cloud to sync data between mobile devices and the server. Main advantage is, that we can build Apps in a fraction of time, without juggling with synchronization.

One of our main challenges was the disability to copy data from one realm to another. But there is still a good solution for this, even though you have to do a lot of coding for that.

Motivation

There are a lot of reasons, why copying realms is a really neat feature. First of all would be versioning your data. But it’s also very neat, when you have several instances (for development, testing etc.) to copy the whole data when you go productive or need some test data in development.

Challenge 1: generating the Schema

When we copy one realm to another, we actually don’t know the used types. We could include a library which contains the types, but then our “RealmCopy” will only work for a specific realm, which is really baloney. That’s as if you develop a copy-program, that can only copy text-files.

So we need to read the source realm, get the schema and create the same schema in the destination. In .Net we can open a realm dynamically, which is pretty cool, because this gives us a chance to get the information we need. Only problem – we can not create or change a schema on a dynamically opened realm.

Solution? We can create dynamic types, and open the destination realm, this will automatically create a schema on the destination realm. (It’s really a little bit awkward, but that’s currently the only chance in .Net – with Javascript, we can mutate the realm directly, but that’s probably due to the nature of that language)

RealmCopy

Open a realm as dynamic:

var username = "";
var password = "";
var realmUrl = "";
var realmName = "TestRealm";
var user = await User.LoginAsync(Credentials.UsernamePassword(username, password, false), new Uri(realmUrl));

var configuration = new FullSyncConfiguration(new Uri(realmName, UriKind.Relative), user)
{
IsDynamic = true
};
var srcRealm = await Realm.GetInstanceAsync(configuration);

now we can go through the scheme and create types:

var types = new List();
foreach (var scheme in schemasToProcess)
{
// create an assembly for our type
var assemblyName = new AssemblyName($"SomeAssemblyName{name}");
var assemblyBuilder = AssemblyBuilder.DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Run);
// create an module for the type
var moduleBuilder = assemblyBuilder.DefineDynamicModule("MainModule");
// get the typebuilder as
var typeBuilder = moduleBuilder.DefineType(name,
TypeAttributes.Public |
TypeAttributes.Class
, parent);
// add properties to type
foreach (var prop in schema)
{
var attributes = new List();
Type propType;
// woven property is needed, so Realm will add it to the destination schema
attributes.Add(typeof(WovenPropertyAttribute));
if (prop.IsPrimaryKey)
{
// add primary key attribute
attributes.Add(typeof(PrimaryKeyAttribute));
}
else if (prop.IsIndexed)
{
// add indexed attribute
attributes.Add(typeof(IndexedAttribute));
}

// get the property type
propType = prop.GetPropertyType(knownTypes);

// set required attribute, when its not nullable
if ((prop.Type & PropertyType.Nullable) != PropertyType.Nullable)
{
attributes.Add(typeof(RequiredAttribute));
}

// create the property
MyTypeBuilder.CreateProperty(typeBuilder, prop.Name, propType, attributes.ToArray());
}
types.Add(typeBuilder.CreateType());
}

In one of my previous blog posts I mentioned how to create types using the TypeBuilder, so I won’t code it out here.
The function GetPropertyType is an extension, it’s just a mapping of the Realm-PropertyType to a CLR type.
Now we can open the destination Realm:

var configuration = new FullSyncConfiguration(new Uri(realmName, UriKind.Relative), user);
configuration.ObjectClasses = types;
var destRealm = await Realm.GetInstanceAsync(configuration);

Challenge 2: copying the realm

This doesn’t actually seem to be a challenge. We go through all objects in source realm and copy it to a new RealmObject, which is created in destination. You can actually use our generated types to open the source realm and use straight reflection for mapping all properties. But be careful, if you did something wrong, you would probably mutate the source realm (what you might not want). You can stick to the dynamic Realm instead and use CallSiteCache to read the properties and reflection to set it to the destination object. (more infos)

Challenge 3: cascaded objects

An object may not only reference atomic types, but also have other RealmObjects as properties. It can even contain cycles. This is really tricky, I solved it by initially creating a TypeBuilder for each SchemaItem, then we can use this TypeBuilder as a PropertyType, without building the actual type. Creating objects can be done recursively, but keep track of the objects, you already created, so you can avoid cycles.

If you have any questions, feel free to ask. ..have fun coding.

using enums with Realm

Realm is a very sweet object-oriented database engine for various languages. Since I am a .Net developer, I’ll stick to the features and problems we have using it within Xamarin Apps.

Recently we encountered a problem, using enums within a RealmObject. The most challenging thing in using Realm is, the constraint that comes with the Realm Models, they can not inherit each other, and Properties may only be a hand full of types or other RealmObjects (see: Realm: Supported Types). For a complete reference to Realm, feel free to stick to the .Net Tutorial on realm.io, it’s pretty neat.

So, as you can imagine from the headline, there is no out-of-the-box support for enums, but it’s pretty easy to achieve a proper solution.

Let’s assume we have Model with some enums:

public class Transaction : RealmObject
{
[PrimaryKey]
public int TransactionId { get; set; }
// this will not compile
public TransactionState State { get; set; }
public string Subject { get; set; }
public double Amount { get; set; }
}
public enum TransactionState
{
Open,
Comitted,
InTransaction,
Aborted,
Failed
}

The problem with this code snippet is, that enums are not supported in Realm, so the compiler (actually Fody) will give us an error, stating it is not a supported type:

Error Fody/Realm: Transaction.State is an ‘AsyncViewModel.Models.TransactionState‘ which is not yet supported. Models\Transaction.cs 12

How to solve this problem? Enums are always backed by another type, usually, it is an int (one can specify other types, but this is normally not the case). How about saving the int to Realm? But we want to keep the actual enum within the model, so we use a pseudo property (since Realm can only save auto-properties, no fields).

public TransactionState State
{
get { return (TransactionState)StateId; }
set { StateId = (int)value; }
}
public int StateId { get; set; }

So that’s it. We could also add an [Ignored] Attribute to the State property, but Fody is automatically ignoring non-auto properties.