Constraint Problem Structure

QUICK! DEFINE: directional arc consistency, constraint tree-like graph, topological sort!

To solve a tree-structured CSP, pick any variable to be the root, then choose an ordering of variables such that each variable appears after its parent in the tree (topological sort).

Any tree with \(n\) nodes has \((n-1)\) edges, so it can be made arc-consistent with \(O(n)\) steps, each of which compares up to \(d\) possible domain values for two variables, giving a total time of \(O(nd^2)\).

Tree-like graph and possible ordering

Tree-CSP Solver Pseudocode

Now that we’ve gotten a sufficiently big hammer, let’s see if we can find some nails!

Cutset Conditioning

One way we can reduce a constraint graph to a tree is by “removing” a node, by assigning value to some vaiables such that the remaining variables form a tree.

Graph to forest of two trees

This works well for binary CSPs, but gets more complicated with higher-order constraints.

In the general case (not in the special case of map coloring), our first chosen value could be wrong! So we have to try each possible value in turn.

Choose a subset \(S\) of the CSP’s variables such that the constraint graph becomes a tree after the removal of \(S\). \(S\) is called a cycle cutset
For each possible assignment to the variables in \(S\) that satisfies all constraints on \(S\),
1. remove from the domains of the remaining variables any values that are inconsistent with the assignment for \(S\)
2. if the remaining CSP has a solution, return it together with the assignment for \(S\)

If the cycle has a cutset of size \(c\), then the total runtime is \(O(d^c\cdot (n-c)d^2)\)… we have to try each of the \(d^c\) combination of values for the variables in \(S\), solving a tree problem of size \(n-c\).

If the graph is almost a tree already, then \(c\) will be small and the savings (over straight backtracking) will be large!

With our 100-boolean example, a cutset of \(c=20\) brings the cost from the lifetime of the universe to a few minutes.

Worst case, \(c\) can be as large as \(n-2\)…

Finding the smallest cycle cutset is NP-hard, but there are a few good algorithms exist.

This algorithmic treatment is called cutset conditioning and comes up in Ch. 13

Tree decomposition

Another way to reduce our constraint graph to a tree is constructing a tree decomposition, a tree where each node contains of a set of variables.

A tree decomposition must satisfy these requirements:

Every variable must appear in at least one node
If two variables are connected by a constraint, they must appear together (with the constraint!) in at least one of the tree nodes.
If a variable appears in two nodes in the tree, it must appear in every node along the path connecting the two

The first two rules ensure that we consider everything, every variable and constraint. The third rule seems a bit less obvious, but it ensures that we have value consistency.

Tree Decomposition of Australia

Once we have our tree-structured graph, we apply \(TreeCSPSolver\) and get a solution in \(O(nd^2)\) time, where \(n\) is the number of tree nodes and \(d\) is the size of the largest domain.

Consider the top left node:, the subproblem has the variables \(\{WA,NT,SA\}\) with the domain \(\{red,green,blue\}\) and constraints \(WA\ne NT, NT\ne SA, SA\ne WA\). But at “tree level” we can consider it a single variable: \(SANTWA\), who’s value is a tuple of three colors.

In this way, we can solve in \(O(nd^2)\) using \(TreeCSPSolver\), which is pretty good as long as…

Revisiting our 100 boolean example, if each node has 10 variables, then \(d=2^{10}\), which would take seconds. But if there is a node with 30 variables, it would take centuries.

Since there may be many ways to decompose a graph into a tree, the trick is making the subproblems as small as possible (putting everything into one node is still a “tree” after all).

The tree width is the size of the largest node (in the decomposition) minus one. The tree width of the actual graph is the minimum width of all its decompositions.

Given a tree of width \(w\), then the problem may be solved in \(O(d^c\cdot(n-c)d^2)\). including the actual tree decomposition. Which is to say… CSPs with constraint graphs of bounded tree size are solvable in polynomial time

But… finding the minimal tree decomposition is NP-hard (heuristics do exist).

So… which is better, the cutset decomposition of time \(O(d^c\cdot (n-c)d^2)\) or the tree decomposition in \(O(nd^{w+1})\)?

When there is a cutset of size \(c\) there is also a tree width of size \(w\lt c+1\), and sometimes may be far smaller! In terms of time, tree decomposition is the way to go.

However, the cycle-cutset approach can be implemented with linear memory, tree decomposition memory usage is exponential in \(w\).

Value Symmetry

So far, we’ve looked at the structure of the graph, but there can also be structure in the values or even in the constraints themselves!

Consider coloring a map with \(d\) colors:

For every consistent solution, there is a set of \(d!\) solutions which can be formed by permuting the color names. e.g. \(WA,NT,SA\) must all have different colors, but there are \(3!=6\) ways to assign the colors.

This is called value symmetry. We would like to reduce the search space by a factor of \(d!\) by breaking this symmetry in assignments.

To do this, we can introduce a symmetry-breaking constraint. One example could be the constraint \(NT<SA<WA\), which requires the variables to be in alphabetical order. What would be the only valid assignment in this case?

In the case of map-coloring, it’s easy to find a symmetry-breaking constraint, however, it’s NP-hard to eliminate all symmetry. Though if you can, it can save you a great deal of work!