Multi-tenant Database Demo for Ninja Web Framework

I have just written a short demo on how to support multi-tenant database for Ninja Web Framework.

The full source code can be downloaded from github.

Features of this demo

  • Support multi-tenant databases using Ninja web framework. Majority of the existing Ninja web framework documentation for database access is still applicable; only small amount of changes is required to provide tenant database information, and to bind the EntityManager provider to each tenant at runtime.
  • Existing database migration scripts can be used to migrate database changes to all tenant databases.
  • Use HikariCP connection pool in Ninja web framework.

Getting tklbam to install on Ubuntu 14.04

I use tklbam from TurnKeyLinux to backup a number of Linux nodes on DigtalOcean and Amazon AWS.

However, tklbam support was broken on Ubtuntu 14.04 where you can't install the tklbam software.

Below is my workaround to get tklbam to install on Ubuntu 14.04.

As described on TurnKeyLinux web page, run the following command to setup the repository.

wget -O - -q \
https://raw.github.com/turnkeylinux/tklbam/master/contrib/ez-apt-install.sh \
| PACKAGE=tklbam /bin/bash

Then go to /etc/apt/sources.list.d and use your favourite text editor to edit the archive.turnkeylinux.org.list. Replace the term jessie with wheezy. Your file should look like this:

deb http://archive.turnkeylinux.org/debian wheezy main

Now run apt-get update to update the packages and then apt-get install tklbam to install tklbam.

My Experience with Using CLIPS

(A chinese translation of this post is available here)

I had the opportunity to incorporate a CLIPS expert system in one of my recent projects for bill plans logics and monitoring of health of our system modules. Both of them are classical sore-points for OO/procedural programming methodologies. We had made an attempt to implement the bill plan logics in an initial version using python, but it ended up in a meshed-up code with half-dozen levels of nested-if-then-else control structures. The system would have eventually ended up to become an excellent case study for project maintenance failure if we had persisted in using an OO/procedural language for the implementation.

Even though CLIPS has worked well for us, I still feel that it may not be suitable for all projects.

Requires radical change in programming paradigm.

Instead of executing your operations in a procedural manner, you have to "train" yourself to re-think your operations as separate rules; these rules operate in tandem in a series of recognize-act cycles. When a group of rules matches, a series of actions can be performed which may change the conditions of the rules in some manner. This may lead to other rulesets (or even the current ruleset) being triggered.

Another issue you would face is how to get these rulesets to "fire" in the order that you want without imposing too much restrictions on them. The conditions in each ruleset should not depend on previous states of other rulesets, i.e. they should not be coupled to each other.

Requires deep understanding of the knowledge domain.

To model the rules, you will need to arrange your knowledge on how things operate in your problem domain into a series of rules/patterns. Usually, you can attempt to perform this knowledge modeling process yourself, or to engage a knowledge modeling expert to assist you.

Execution should be data-driven or pattern-driven.

If you encounter a need to perform lots of if-conditional statements on many variables, the execution process is most likely data-driven or pattern-driven. Any non-trivial knowledge domain usually involves more than a dozen variables to work with. In OO/procedural, this means your source code will end up in meshed-up manner with multi-levels of nested if-else statements. CLIPS language syntax allows you to specify the conditions (i.e. rules) for a group of tasks to be executed in an organized manner.

However, one must be careful to plan and organize the data structure of the CLIPS facts and classes. Every datum in each CLIPS class and facts should be well-encapsulated using the same data-encapsulation principles in OO development methdology).

CLIPS专家系统实现经验

此贴原先以英文语言刊登在此处

本人在前些时候有机会推行一项结合CLIPS专家系统的网上订购和实时系统监控的项目。这两样功能无论是以过程语言或是面向对象语言来实现都有相当大的难度。主要是当逻辑因子和监控变量增加时,要评价这些变量的复杂性将会以指数增长的方式变得极其复杂。我们尝试以Python语言来实现这两样功能,可是初步实现这两样功能的源码出现六层次的嵌套IF-ELSE结构,而最终得被迫放弃。若我们坚持以Python来继续实现,想必将来的软件维护将会是一项艰巨的任务。

虽然CLIPS专家系统最终为我们解决了评价变量复杂性的问题,但是本人还是觉得CLIPS专家系统并不适用于所有的逻辑问题。

CLIPS编程范式需要激进的思想转变

必须放弃以过程方式来思考如何实现各个评价操作,各个操作须以个别规则来实现,而这些规则要以识别动作周期的方式在CLIPS环境里运行。当一组的规则匹配成功时,这将引发相关的动作指令。这些指令可能修正这些规则的一些先前条件,而匹配其他(甚至是原先)的规则。另一项值得注意的是要如何设计这些规则,让这些规则能按照您想要的顺序来引发,而要在同时避免添加太多的约束。各个规则的引发条件应该避免依赖其他规则的状态,也就是规则之间的耦合度应该降到最低点。

要有深层的领域知识

在编写CLIPS规则时,您要将您领域里的各个事物运行的知识整理成一系列的规则和模式。您可以尝试进行这一个过程,或是聘请专家来协助您。

执行应该是数据驱动或是模式驱动

当您需要以多个IF语句来推理多个变量时,这执行过程很可能是数据驱动或是模式驱动的。任何有一定规模的领域都会牵涉到十几个或更多的变量,这意味着您的过程或面向对象源码将会呈现网格结构或是包含多层次的嵌套IF-ELSE结构。CLIPS的语言规范能让您针对各组指令而设置个别的引发条件,但是要注意对CLIPS事实和类的规划,CLIPS事实和类的属性应该遵循数据封装的原理来设计。

Scanning Network for Windows Shared Folders

I have noticed that there are quite a few requests for a network scanning application to search for Windows shared folders.

I thought this will be an interesting implementation for pysmb (a pure Python implementation of the SMB1 and SMB2 protocol that supports the Windows file sharing functionality) as well as for developers who are learning to use pysmb for their applications.

You will need Python 2.4 and above, and have installed pyasn1 and pysmb. Next, download the ScanNetworkForSMB.py script. Then, run the script with 1 IP address to scan for a single machine, or a start/end IP address pair to scan a range of IP addresses. If the scanned machine has its Windows sharing port (port 137) active, the script will print out its IP address together with the names associated with this machine.

Sample output

$bash> python ScanNetworkForSMB.py 192.168.1.2 192.168.1.256
Beginning scanning 255 IP addresses...

     192.168.1.2 --> CETUS WORKGROUP
   192.168.1.109 --> I7PC WORKGROUP

Query timeout. No replies from 253 IP addresses