Inhomogeneous lattice dynamical systems and the boundary effect

Abstract

This study considers the dynamics of cellular neural network-based inhomogeneous lattice dynamical systems (CNN-based ILDS). The influence of three kinds of boundary conditions, say, the periodic, Dirichlet, and Neumann boundary conditions, is elucidated. We reveal that the complete stability of CNN-based ILDS and, under some prescriptions, the topological entropies of CNN-based ILDS with/without the boundary condition are identical.

MSC:37B10.

1 Introduction

In the past few decades, the standard cellular neural networks (CNNs) introduced by Chua and Yang [1] have been one of the most investigated paradigms for neural information processing [2]. In a wide range of applications, the CNNs are required to be completely stable, i.e., each trajectory should converge toward some stationary state. In the study of stationary solution, the investigation of mosaic solutions is most essential in CNNs due to the learning algorithm and training processing. More abundant output patterns make the learning algorithm more efficient. Mathematically, the study of the mosaic solutions is reasonable due to the following two facts: (1) complete stability of a wide range of parameters, and (2) the output function of CNNs is a piecewise linear function with constant value for $|x|\ge 1$; namely,

$$f(x)=\frac{1}{2}(|x+1|-|x-1|).$$

The outputs $y=(f(x_i))$, called patterns, are essential for understanding CNN systems. Traditionally, the template for CNNs is homogeneous (also known as isotropic), i.e., the template is space-invariant. However, there are more and more CNNs using inhomogeneous templates to describe some of the problems that arise from the biological and ecological contexts [3–8], skeletonization [9], image processing [10, 11], artificial locomotion control [12], and delayed-type CNN [13–16]. Some new and interesting phenomena of pattern formation and spatial chaos were also found in inhomogeneous multi-layer neural networks. In this paper, the entropy with/without the boundary effect for stable patterns of inhomogeneous CNN is investigated. Entropy is a quantity used for measuring the complexity of the output patterns and it plays an important role in learning algorithm. Surprisingly, such a topic reveals the deep connection with symbolic dynamical systems (SDS). In 1-d CNN, it has been proved that the space of the mosaic solutions (defined later) forms a 1-d subshift of finite type (SFT, [17]). Recently, it has also been proved that the mosaic solutions of a multi-layer CNN (MCNN) form a sofic space[18–20], which is a factor of SFT. The mosaic solutions of inhomogeneous CNN, indeed, produce new shift spaces in SDS. To clarify the investigation of inhomogeneous CNNs, we concentrate our discussion on two classes, and the methodology can be applied in a general case. More specifically, two types of inhomogeneous CNN, constant and arithmetic CNN, are presented herein. It is proved that the space of the mosaic solutions forms a new class in SDS (Theorem 2.10 and Theorem 3.5), called a multiple shift space, which was initiated from the study of the arithmetic regression property in the number theory of mathematics [21–24]. The complexity (topological entropy) can be computed due to the equivalence of the mosaic solutions and multiple shift spaces (Theorem 2.13 and Theorem 3.7). The positivity of entropy unveils the spatial chaos for given systems and pattern formation for zero entropy. Such topics, e.g., pattern formation or synchrony phenomena on LDS, have been investigated by many mathematicians and physicists [25–30].

Besides the entropy formula being established, the boundary effect for constant CNNs and arithmetic CNNs are also considered. Three types of boundary conditions, periodic, Dirichlet, and Neumann, are proposed to a given constant CNN and arithmetic CNN. Sufficient conditions are found for the preservation of entropy under the boundary constraint (Theorem 2.13 and Theorem 3.7), i.e., $h_P=h_D=h_N=h$. This extends the results in the classical CNNs (cf.[31, 32]). The preservation of entropy under the boundary constraint is unavoidable [33]; since the number of nodes in a lattice is infinite, one usually uses the finite approximation method to exploit the statistical properties of the whole lattice.

Some related topics are also addressed herein. It is known that the mosaic solution of single/multi-layer template-invariant CNNs is constrained by the so-called separation property, namely, not all but some of the patterns that satisfy this property will appear as the mosaic solution for a given CNN [34]. However, more combinations of mosaic patterns will help the learning and training process to be more efficient. It is believed that the template-variant or the multi-layer CNN will achieve this goal. In mathematical language, it means that $h(\mathbb{T})$ will be ϵ-dense in $[0,log2]$ when parameter runs all of the parameter space, where $h(\mathbb{T})$ denotes the entropy function according to the parameter . It is proved that constant CNNs possess the ϵ-dense property (Theorem 2.14), and it seems that arithmetic CNNs also satisfy the ϵ-dense property by numerical computation (Conjecture 3.8). We believe that further interesting applications of the results presented (or of the generalizations) can be obtained.

We organize the material in this paper as follows. Section 2 introduces the concepts of general inhomogeneous CNN-based LDS and constant-type multiple CNNs. Stability, partition of the parameter space and the equivalence of mosaic solutions with a multiple shift space are discussed therein. This together with the exact number of mosaic solutions under the boundary constraint (Lemma 2.12) is used to derive the entropy formula and entropy preservation property. Parallel discussions for arithmetic-type multiple CNNs can be found in Section 3. Some one- and two-dimensional examples are addressed in Section 4, and we leave the discussion in Section 5.

2 Constant cellular neural networks

In this section, we investigate a specified type of inhomogeneous LDS named constant-type multiple cellular neural network (constant CNN). To clarify the elucidation, Section 2.1 concentrates on the constant CNNs with nearest neighborhood. The general cases of constant CNNs and deeper architecture are investigated in the rest of this section.

2.1 Constant cellular neural networks with nearest neighborhood

First we consider the LDS realized as

$$\{\begin{array}{l}\frac{d}{dt}{x}_{2i+1}(t)=-x_{2i+1}(t)+z_o+{\sum }_{|k|\le 1}{a}_{k;o}f(x_{2(i+k)+1}(t)),\\ \frac{d}{dt}{x}_{2i}(t)=-x_{2i}(t)+z_e+{\sum }_{|k|\le 1}{a}_{k;e}f(x_{2(i+k)}(t))\end{array}$$

(1)

for $i\in \mathbb{Z}$. Denote the parameters that relate to the odd and even positions by ${\mathbb{T}}_o=[a_{-1;o},a_{0;o},a_{1;o}]$ and ${\mathbb{T}}_e=[a_{-1;e},a_{0;e},a_{1;e}]$, respectively. We call $\mathbb{T}=[{\mathbb{T}}_o,{\mathbb{T}}_e]$ the feedback template of (1), and $\mathbf{z}=[z_o,z_e]$ is the threshold. It is seen that the templates in (1) are periodic; the prescribed model is a generalization of the classical cellular neural network and is called the constant-type multiple cellular neural network.

A system of ordinary differential equations is called completely stable if each of its solution x approaches an equilibrium state. Let ${\mathbf{x}}_o$, ${\mathbf{x}}_e$ denote the collection of cells in odd and even coordinates, respectively. Express (1) as

$${\dot{\mathbf{x}}}_c=-{\mathbf{x}}_c+{\mathbf{A}}_{c}F({\mathbf{x}}_c)+{\beta }_c,$$

(2)

where $c=o,e$, ${\mathbf{x}}_c\in {\mathbb{R}}^n$, $F({\mathbf{x}}_c)\in {\mathbb{R}}^n$ is a diagonal mapping (herein $F({\mathbf{x}}_o)={(f(x_1),\dots ,f(x_{2n-1}))}^t$ and $F({\mathbf{x}}_e)={(f(x_2),\dots ,f(x_{2n}))}^t$), and ${\beta }_c={(z_c,\dots ,z_c)}^t\in {\mathbb{R}}^n$. The sufficient conditions for the complete stability of (1) are given as follows. The extension of Theorem 2.1 can be seen in Theorem 2.5.

Theorem 2.1 A constant CNN is completely stable if, for$c=o,e$, one of the following conditions is satisfied.

S1 ${\mathbb{T}}_c$is symmetric.

S2 ${\mathbf{A}}_c(i,i)>1$and$K_c^{-1}(i,j)\ge 0$for all i, j, where

$$K_c(i,j)=\{\begin{array}{ll}{\mathbf{A}}_c(i,i)-1,& i=j;\\ -|{\mathbf{A}}_c(i,j)|,& \mathit{\text{otherwise}}.\end{array}$$

The complete stability of (1) demonstrates that the investigation of the equilibrium solutions is essential. To make the discussion more clear, we focus on the mosaic solutions, i.e., $|x_i|>1$ for all i, and study the complexity of the output space $\mathbf{Y}=\{(y_i):y_i=f(x_i),i\in \mathbb{Z}\}$ of the mosaic solutions. We investigate the complexity of the output space in two aspects:

• ${\mathrm{\Gamma }}_n(\mathbf{Y})$: The exact number of patterns of length n.

• $h(\mathbf{Y})$: The topological entropy of the output space.

To achieve our target, we introduce the ordering matrix and transition matrix first. The ordering matrix is defined as

herein the pattern ‘−’ stands for the state $y_i=-1$ and ‘+’ stands for $y_i=1$. Let

$${\mathbf{Y}}_o=\{(y_{2i+1}):(y_i)\in \mathbf{Y}\}\text{and}{\mathbf{Y}}_e=\{(y_{2i}):(y_i)\in \mathbf{Y}\}.$$

For $c=o,e$, define the transition matrix $T_c$ of ${\mathbf{Y}}_c$ by

$$T_c(i,j)=\{\begin{array}{ll}1,& \mathbb{X}(i,j)\in {\mathrm{\Sigma }}_3({\mathbf{Y}}_c);\\ 0,& \text{otherwise},\end{array}$$

where ${\mathrm{\Sigma }}_n(X)$ consists of patterns of length n in X. Yielding $T_o$ and $T_e$, we derive the formula of ${\mathrm{\Gamma }}_n(\mathbf{Y})$ and $h(\mathbf{Y})$. For the general cases of constant CNNs, Theorem 2.2 is generalized by Lemma 2.11 and Theorem 2.13.

Theorem 2.2 Suppose$n=2k+r$for some$k\ge 3$, $r=0,1$. $T_o$and$T_e$are the transition matrices of${\mathbf{Y}}_o$and${\mathbf{Y}}_e$, respectively. Then

$${\mathrm{\Gamma }}_n(\mathbf{Y})=\parallel T_o^{k+r-2}\parallel \cdot \parallel T_e^{k-2}\parallel ,$$

where$\parallel A\parallel ={\mathrm{\Sigma }}_{i,j}|A(i,j)|$for any nonnegative matrix$A=(A(i,j))\in {\mathbb{R}}^{m\times n}$. Moreover, the topological entropy of Y is

$$h(\mathbf{Y})=\frac{1}{2}log({\rho }_o\cdot {\rho }_e),$$

where${\rho }_o$and${\rho }_e$are the spectral radii of$T_o$and$T_e$, respectively.

In the meantime, it is natural to elucidate the influence of boundary conditions on the exact number of patterns of length n and topological entropy. Three types of boundary conditions, periodic, Neumann, and Dirichlet boundary conditions, are considered. To reflect the influence of the boundary conditions, we introduce three boundary matrices. Let

$$\begin{array}{c}{E}^{(o)}=\left(\begin{array}{cccc}1& 0& 1& 0\\ 1& 0& 1& 0\\ 1& 0& 1& 0\\ 1& 0& 1& 0\end{array}\right),E^{(e)}=\left(\begin{array}{cccc}0& 1& 0& 1\\ 0& 1& 0& 1\\ 0& 1& 0& 1\\ 0& 1& 0& 1\end{array}\right),\hfill \\ E^{(u)}=\left(\begin{array}{cccc}1& 1& 1& 1\\ 1& 1& 1& 1\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right),E^{(l)}=\left(\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 1& 1& 1& 1\\ 1& 1& 1& 1\end{array}\right).\hfill \end{array}$$

The periodic boundary matrix $R^P$ is a $16\times 16$ matrix defined by

$$R^P=\left(\begin{array}{cccc}{E}^{(o)}& E^{(o)}& E^{(o)}& E^{(o)}\\ E^{(o)}& E^{(o)}& E^{(o)}& E^{(o)}\\ E^{(e)}& E^{(e)}& E^{(e)}& E^{(e)}\\ E^{(e)}& E^{(e)}& E^{(e)}& E^{(e)}\end{array}\right).$$

The Neumann boundary condition infers zero flux on both sides of the space. The left and right Neumann boundary matrices are then defined by

$$L^N=\left(\begin{array}{cccc}{E}^{(u)}& E^{(u)}& E^{(u)}& E^{(u)}\\ E^{(u)}& E^{(u)}& E^{(u)}& E^{(u)}\\ E^{(l)}& E^{(l)}& E^{(l)}& E^{(l)}\\ E^{(l)}& E^{(l)}& E^{(l)}& E^{(l)}\end{array}\right)\text{and}{R}^N=\left(\begin{array}{cccc}{E}^{(o)}& E^{(e)}& E^{(o)}& E^{(e)}\\ E^{(o)}& E^{(e)}& E^{(o)}& E^{(e)}\\ E^{(o)}& E^{(e)}& E^{(o)}& E^{(e)}\\ E^{(o)}& E^{(e)}& E^{(o)}& E^{(e)}\end{array}\right),$$

respectively. Furthermore, the Dirichlet boundary condition indicates that both sides of the space are constant states and the corresponding boundary matrices are

$$\begin{array}{c}{L}^{D_{-}}=\left(\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right),R^{D_{-}}=\left(\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 0\end{array}\right),\hfill \\ L^{D_{+}}=\left(\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right),R^{D_{+}}=\left(\begin{array}{cccc}0& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 1\end{array}\right).\hfill \end{array}$$

Herein $D_{-}$ and $D_{+}$ relate to states ‘−’ (i.e., $y_0=y_{n+1}=-1$) and ‘+’ (i.e., $y_0=y_{n+1}=1$), respectively. Before presenting the formula of ${\mathrm{\Gamma }}_n^B(\mathbf{Y})$ and $h_B(\mathbf{Y})$ under the boundary condition $B=P,N,D_{-},D_{+}$, we introduce two operations of matrices.

Definition 2.3

• 1. Suppose that $A\in {\mathcal{M}}_{p\times q}(\mathbb{R})$ is a $p\times q$ matrix and $B\in {\mathcal{M}}_{r\times s}(\mathbb{R})$ is an $r\times s$ matrix. The Kronecker product $A\otimes B\in {\mathcal{M}}_{pr\times qs}$ is defined by

  $$A\otimes B={(A(i,j)B)}_{1\le i\le p,1\le j\le q}.$$

• 2. Suppose that $A,B\in {\mathcal{M}}_{p\times q}(\mathbb{R})$ are $p\times q$ matrices. The Hadamard product $A\circ B\in {\mathcal{M}}_{p\times q}(\mathbb{R})$ is defined by

  $$A\circ B={(A(i,j)B(i,j))}_{1\le i\le p,1\le j\le q}.$$

With the introduction of the boundary matrices and the Kronecker and Hadamard products, we obtain Theorem 2.4 which reveals the formulae of exact number of patterns and topological entropy under the influence of three kinds of boundary conditions. The extension of Theorem 2.4 for general constant CNNs is demonstrated by Lemma 2.12 and Theorem 2.13.

Theorem 2.4 Suppose$n=2k+r$for some$k\ge 3$, $r=0,1$. $T_o$and$T_e$are the transition matrices of${\mathbf{Y}}_o$and${\mathbf{Y}}_e$, respectively. Then$h_B(\mathbf{Y})=h(\mathbf{Y})$, $B=P,N,D_{-},D_{+}$, if$T_o$and$T_e$are primitive matrices. Furthermore, the exact number of patterns of length n with boundary condition$B=P,N,D_{-},D_{+}$are as follows:

• The periodic boundary condition:

  $${\mathrm{\Gamma }}_n^P(\mathbf{Y})=\{\begin{array}{ll}\parallel (T_o^{k-1}\otimes T_o)\circ R^P\parallel \cdot \parallel T_e^{k-2}\parallel ,& r=0;\\ \parallel (T_o^{k-1}\otimes T_e^{k-1})\circ R^P\parallel ,& r=1.\end{array}$$

  (3)

• The Neumann boundary condition:

  $${\mathrm{\Gamma }}_n^N(\mathbf{Y})=\{\begin{array}{ll}\parallel L^N\circ (T_o^k\otimes T_e^{k-1})\circ R^N\parallel ,& r=0;\\ \parallel ((L^N\circ (T_o^k\otimes T_e^{k-1}))\otimes T_e^{k-1})\circ (E_2\otimes R^N)\parallel ,& r=1.\end{array}$$

  (4)

Herein$E_2=\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)$.

• The Dirichlet boundary condition:

  $${\mathrm{\Gamma }}_n^B(\mathbf{Y})=\{\begin{array}{ll}\parallel (L^B{T}_e^{k-1})\otimes (T_o^{k-1}{R}^B)\parallel ,& r=0;\\ \parallel T_o^{k-1}\parallel \cdot \parallel L^B{T}_e^k{R}^B\parallel ,& r=1.\end{array}$$

  (5)

Herein$B=D_{-},D_{+}$relate to the conditions that the patterns on the boundary are ‘−’ and ‘+’, respectively.

2.2 Stability of constant cellular neural networks

The rest of this section extends the results in Section 2.1. To make the paper compact, we introduce the general setting for multi-dimensional inhomogeneous LDS and then concentrate on the one-dimensional case. The elucidation of multi-dimensional systems will be investigated in another paper.

A D-dimensional inhomogeneous CNN-based LDS is realized as

$$\frac{d}{dt}{x}_{\mathbf{i}}(t)=-x_{\mathbf{i}}(t)+z_{\mathbf{i}}+\sum _{\mathbf{k}\in {\mathcal{N}}_{\mathbf{i}}}{a}_{\mathbf{k};\mathbf{i}}f(x_{\mathbf{k}}(t)),$$

(6)

where $\mathbf{i}\in {\mathbb{Z}}^D$, and ${\mathcal{N}}_{\mathbf{i}}$, which is a finite subset of ${\mathbb{Z}}^D$, indicates the neighborhood for neuron $x_{\mathbf{i}}$. The piecewise linear function $f(x)=\frac{1}{2}(|x+1|-|x-1|)$ is called the output function; $\mathbf{z}=[z_{\mathbf{i}}]$ refers to the threshold, and the feedback template$\mathbb{T}={[{\mathbb{T}}_{\mathbf{i}}]}_{\mathbf{i}\in {\mathbb{Z}}^D}$ stores the weight of local interaction between neurons, where ${\mathbb{T}}_{\mathbf{i}}={[a_{\mathbf{k};\mathbf{i}}]}_{\mathbf{k}\in {\mathcal{N}}_{\mathbf{i}}}$.

An inhomogeneous CNN-based LDS is called a constant CNN if the neighborhood , the template , and z are periodic up to shifts. More precisely, there exists $\ell \in \mathbb{N}$ such that ${\mathcal{N}}^{\prime }=\{{\mathcal{N}}_{\mathbf{i}}^{\prime }:\mathbf{i}\in {\mathbb{Z}}^D\}$, ${\mathbb{T}}^{\prime }={[{\mathbb{T}}_{\mathbf{i}}^{\prime }]}_{\mathbf{i}\in {\mathbb{Z}}^D}$, and $\mathbf{z}={[z_{\mathbf{i}}]}_{\mathbf{i}\in {\mathbb{Z}}^D}$ satisfy ${\mathcal{N}}_{\mathbf{i}+\mathbf{j}\ell }^{\prime }={\mathcal{N}}_{\mathbf{i}}^{\prime }$, ${\mathbb{T}}_{\mathbf{i}+\mathbf{j}\ell }^{\prime }={\mathbb{T}}_{\mathbf{i}}^{\prime }$, and $z_{\mathbf{i}+\mathbf{j}\ell }=z_{\mathbf{i}}$ for $\mathbf{i},\mathbf{j}\in {\mathbb{Z}}^D$, where

$${\mathbf{K}}_{\mathbf{i}}^{\prime }={\mathbf{K}}_{\mathbf{i}}-\mathbf{i}=\{\mathbf{j}-\mathbf{i}:\mathbf{j}\in {\mathbf{K}}_{\mathbf{i}}\},\mathbf{K}=\mathcal{N},\mathbb{T}.$$

It is seen that the constant CNNs generalize the concept of the classical CNNs that were introduced in [1, 35]. More precisely, a classical CNN is a constant CNN with $\ell =1$. The essential description of a one-dimensional constant CNN is presented in the following form:

$$\frac{d}{dt}{x}_i(t)=-x_i(t)+z_{\overline{i}}+\sum _{k\in {\mathcal{N}}_{\overline{i}}}{a}_{k;\overline{i}}f(x_{i+\ell \cdot k}(t)),$$

(7)

where $1\le \overline{i}\le \ell$ and . Without loss of generality, we assume ${\mathcal{N}}_i=\{-d,\dots ,0,\dots ,d\}$ for some $d\in \mathbb{N}$, $1\le i\le \ell$. In this case, the feedback template of (7) is $\mathbb{T}={[{\mathbb{T}}_j]}_{1\le j\le \ell }$, where ${\mathbb{T}}_j=[a_{-d;j},\dots ,a_{0;j},\dots ,a_{d;j}]$. A stationary solution $x={(x_i)}_{i\in \mathbb{Z}}$ is called a mosaic solution if $|x_i|>1$ for all $i\in \mathbb{Z}$, and $y={(y_i)}_{i\in \mathbb{Z}}={(f(x_i))}_{i\in \mathbb{Z}}$ is called a mosaic pattern. A system of ordinary differential equations is said to be completely stable if every trajectory tends to an equilibrium point. Theorem 2.5 infers that a constant CNN is a completely stable system. (We remark that Theorem 2.5 is an extension of Theorem 2.1.)

Theorem 2.5 Suppose that$(\mathbb{T},\mathbf{z})$is the template of (7) and the system is written as

$$\dot{{\mathbf{x}}_j}=-{\mathbf{x}}_j+{\mathbf{A}}_{j}F({\mathbf{x}}_j)+{\beta }_j,1\le j\le n.$$

Then a constant CNN is completely stable if, for$1\le j\le \ell$, one of the following conditions is satisfied.

• (1) ${\mathbb{T}}_j$ is symmetric.

• (2) $K_j$ is nonsingular and $K_j^{-1}\ge 0$, where $K_j$ is defined in (10).

Let $\mathrm{\Lambda }=\{1,\dots ,\ell \}$ be a finite index set. The one-dimensional lattice ℤ can be decomposed into ℓ non-overlapping subspaces

$$\mathbb{Z}=\bigcup _{j\in \mathrm{\Lambda }}{\mathbb{Z}}_j=\bigcup _{j\in \mathrm{\Lambda }}\{m:m=C\ell +j,C\in \mathbb{Z}\}=\bigcup _{j\in \mathrm{\Lambda }}\{j_i,i\in \mathbb{Z}\}.$$

Equation (7) can then be restated as

$$\frac{d}{dt}{x}_{j_i}=-x_{j_i}+z_j+\sum _{|k|\le d}{a}_{k;j}f(x_{j_{i+k}}),j\in \mathrm{\Lambda },i\in \mathbb{Z}.$$

(8)

(It is easily seen that $j_i=j+\ell i$. We reindex the coordinates of neurons to clarify the upcoming investigation.) To prove Theorem 2.5, we consider two kinds of feedback templates separately. For the case that the feedback template of a classical CNN is symmetrical, Forti and Tesi demonstrated that it is completely stable.

Theorem 2.6 ([36])

A classical CNN with symmetric feedback template is completely stable.

For the case that the feedback template is not symmetrical, suppose that a CNN with n-neurons is described as follows:

$$\dot{x}=-x+\mathbf{A}F(x)+\beta ,$$

(9)

where $x\in {\mathbb{R}}^n$, A is an $n\times n$ constant matrix with diagonal elements satisfying

$$\mathbf{A}(i,i)>1,i=1,2,\dots ,n,$$

$F(x)={(f(x_1),f(x_2),\dots ,f(x_n))}^t\in {\mathbb{R}}^n$ is a diagonal mapping from ${\mathbb{R}}^n$ to ${\mathbb{R}}^n$, and $\beta ={({\beta }_1,{\beta }_2,\dots ,{\beta }_n)}^t$ is a constant vector. Takahashi and Chua proposed a criterion to determine whether a CNN is completely stable.

Theorem 2.7 ([37])

Let K be an $n\times n$ matrix satisfying

$$K(i,j)=\{\begin{array}{ll}\mathbf{A}(i,i)-1,& i=j;\\ -|\mathbf{A}(i,j)|,& \mathit{\text{otherwise}}\end{array}$$

(10)

for$1\le i,j\le n$. A classical CNN with asymmetric feedback template is completely stable if K is nonsingular and$K^{-1}\ge 0$, herein a matrix$A\ge 0$means that$A(i,j)\ge 0$for all i, j.

It comes immediately from Theorem 2.7 that if the feedback template $A=[a_{-1},a_0,a_1]$ of a CNN is asymmetric, then the system is completely stable provided there exists a positive constant r such that

$$a_0>r^{-1}{a}_{-1}+r{a}_1.$$

(11)

Proof of Theorem 2.5 Suppose $|\mathrm{\Lambda }|=1$; in this case, a constant CNN is deduced to be a classical CNN. Theorem 2.6 infers that a constant CNN is completely stable if the feedback template is symmetrical. Whenever is asymmetric, the system is still completely stable if the matrix K defined in (10) is nonsingular and $K^{-1}\ge 0$. It is indicated via (8) that a constant CNN can be decomposed into ℓ independent CNN subsystems, the complete stability of a constant CNN comes from the complete stability of every subsystem. □

For a fixed template, the collection of mosaic patterns $\mathbf{Y}=\{y={(y_i)}_{i\in \mathbb{Z}}:y_i=f(x_i),|x_i|>1\}$ is called the output space of (7). Since the neighborhood ${\mathcal{N}}_i$ is finite for each i, the output space is determined by the so-called admissible local patterns. Suppose that y is a mosaic pattern, for each $j\in \mathrm{\Lambda }$ and $i\in \mathbb{Z}$, the necessary and sufficient condition for $y_{j_i}=1$ is

$$a_{0;j}-1+z_j>-\sum _{0<|k|\le d}{a}_{k;j}{y}_{j_{i+k}},$$

(12)

and the necessary and sufficient condition for $y_{j_i}=-1$ is

$$a_{0;j}-1-z_j>\sum _{0<|k|\le d}{a}_{k;j}{y}_{j_{i+k}}.$$

(13)

Set

$$\begin{array}{rl}{\mathcal{B}}_j(+)& =\{\boxed{y_{-d}\cdots y_0\cdots y_d}:y_{-d},\dots ,y_d\in \{-1,1\}\text{ satisfy (12)},y_0=1\},\\ {\mathcal{B}}_j(-)& =\{\boxed{y_{-d}\cdots y_0\cdots y_d}:y_{-d},\dots ,y_d\in \{-1,1\}\text{ satisfy (13)},y_0=-1\}.\end{array}$$

The set of admissible local patterns ℬ of a constant CNN is then

$$\mathcal{B}(\mathbb{T},\mathbf{z})=({\mathcal{B}}_1(+),\dots ,{\mathcal{B}}_{\ell }(+),{\mathcal{B}}_1(-),\dots ,{\mathcal{B}}_{\ell }(-)).$$

Similar to the discussion in [17], the output space Y can be represented as

$$\mathbf{Y}=\{y=(y_{j_i}):y_{j_{i-d}}\cdots y_{j_i}\cdots y_{j_{i+d}}\in ({\mathcal{B}}_j(+),{\mathcal{B}}_j(-))\text{ for }j\in \mathrm{\Lambda },i\in \mathbb{Z}\}.$$

(Recall that in the above equation, $j_i=j+\ell i$.)

One of the important research issues in the circuit theory is the learning problem. That is to say, mathematically, for what and how many phenomena the constant CNNs are capable of exhibiting. Theorem 2.9 infers that once $|\mathcal{N}|=max\{|{\mathcal{N}}_j|:j\in \mathrm{\Lambda }\}$ is fixed, there are finitely many equivalent classes of templates and z so that the basic sets of admissible local patterns $\mathcal{B}(\mathbb{T},\mathbf{z})$ are constrained. Let ${\mathcal{P}}^n=\{(\mathbb{T},z):\mathbb{T}=[a_{-d},\dots ,a_d]\in {\mathbb{R}}^{2d+1},z\in \mathbb{R}\}$ be the parameter space of the classical CNNs, where $n=2d+2$. Theorem 2.8 indicates that the ${\mathcal{P}}^n$ can be partitioned into a finite number of subregions such that each subregion has the same mosaic patterns.

Theorem 2.8 ([34])

There is a positive integer $K(n)$ and a unique set of open subregions ${\{P_k\}}_{k=1}^K$ satisfying

• (i) ${\mathcal{P}}^n={\bigcup }_{k=1}^K{\overline{P}}_k$,

• (ii) $P_i\cap P_j=\mathrm{\varnothing }$ if $i\ne j$,

• (iii) $(\mathbb{T},z)$ and $({\mathbb{T}}^{\prime },z^{\prime })\in P_k$ for some k if and only if $\mathcal{B}(\mathbb{T},z)=\mathcal{B}({\mathbb{T}}^{\prime },z^{\prime })$.

Here$\overline{P}$is the closure of P in$P^n$.

Let $\mathcal{P}=\{(\mathbb{T},z):\mathbb{T}=[{\mathbb{T}}_1,\dots ,{\mathbb{T}}_{\ell }],z=[z_1,\dots ,z_{\ell }]\}$ be the parameter space of (7). The following theorem demonstrates that is also partitioned into a finite number of equivalent subregions.

Theorem 2.9 (Separation property)

There is a positive integer K and a unique set of open subregions ${\{P_k\}}_{k=1}^K$ satisfying

• (i) $\mathcal{P}={\bigcup }_{k=1}^K{\overline{P}}_k$,

• (ii) $P_i\cap P_j=\mathrm{\varnothing }$ if $i\ne j$,

• (iii) $(\mathbb{T},z)$ and $({\mathbb{T}}^{\prime },z^{\prime })\in P_k$ for some k if and only if $\mathcal{B}(\mathbb{T},z)=\mathcal{B}({\mathbb{T}}^{\prime },z^{\prime })$.

Proof Similar to the proof of Theorem 2.5, a constant CNN is reduced to a classical CNN whenever $|\mathrm{\Lambda }|=1$, hence Theorem 2.9 is performed in this case. When $|\mathrm{\Lambda }|\ge 2$, the basic set of admissible local patterns $\mathcal{B}(\mathbb{T},z)$ of (7) is the ordered union of the basic set of admissible local patterns $\mathcal{B}({\mathbb{T}}_j,z_j)$. More specifically, is isomorphic to the direct product ${\prod }_{j\in \mathrm{\Lambda }}{\mathcal{P}}_j$, where ${\mathcal{P}}_j$ is the parameter space of (7) _j , the subsystem of (7) restricting to the cells ${\{x_{C\ell +j}\}}_{C\in \mathbb{Z}}$. Since, for $j=1,\dots ,\ell$, each parameter space ${\mathcal{P}}_j$ is partitioned into a finite number of equivalent subregions by Theorem 2.8, is then the union of a unique set of open subregions ${\{P_k\}}_{k=1}^K$ which satisfies conditions (i) to (iii). This derives the desired result. □

Let $q\ge 2$ be an integer, and let Ω be a subset of the symbolic space ${\mathrm{\Sigma }}_m={\{0,\dots ,m-1\}}^{\mathbb{Z}}$ which is invariant under the shift map $\sigma :{\mathrm{\Sigma }}_m\to {\mathrm{\Sigma }}_m$ defined by $\sigma {(x)}_i=x_{i+1}$. Denote

$$X_{\mathrm{\Omega }}=\{\omega ={(x_k)}_{k=1}^{\mathrm{\infty }}\in {\mathrm{\Sigma }}_m:{(x_{i+qj})}_{j\in \mathbb{Z}}\in \mathrm{\Omega }\text{ for all }i\},$$

which is invariant under σ. The set $X_{\mathrm{\Omega }}$ is called a multiple subshift if Ω is a subshift. Equation (8) together with the proof of Theorem 2.9 asserts that the output space Y of a constant CNN is decomposed into subspaces ${\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_{\ell }$. Observe that Y is topologically conjugated to the direct product of the output spaces ${\mathbf{Y}}_j$ of the classical CNNs, that is, $\mathbf{Y}\cong {\prod }_{j\in \mathrm{\Lambda }}{\mathbf{Y}}_j$, where ${\mathbf{Y}}_j$ is determined by ${\mathcal{B}}_j=({\mathcal{B}}_j(+),{\mathcal{B}}_j(-))$. This derives Theorem 2.10, which indicates that the output space of a constant CNN is a multiple subshift for some parameters.

Theorem 2.10 Given a set of templates$(\mathbb{T},\mathbf{z})$, where$\mathbb{T}=[{\mathbb{T}}_1,\dots ,{\mathbb{T}}_{\ell }]$and$\mathbf{z}=[z_1,\dots ,z_{\ell }]$. Let Y be the solution space of the constant CNN with respect to$(\mathbb{T},\mathbf{z})$. Then

$$\mathbf{Y}\cong X_{\mathrm{\Omega }}$$

if${\mathbb{T}}_i={\mathbb{T}}_j$and$z_i=z_j$for$1\le i,j\le \ell$, where Ω is a SFT that comes from the output space of the classical CNN with respect to template$({\mathbb{T}}_1,z_1)$.

2.3 Boundary effect on constant cellular neural networks

This subsection elucidates the influence of the boundary condition on the exact number of mosaic patterns of finite length and on the growth rate as the length increases. The investigation starts with formulating the number of patterns. Denote by ${\mathbb{Z}}_{n\times 1}=\{k\in \mathbb{Z}:1\le k\le n\}$ the coordinates of the neurons. In this case, the boundary sites are $\mathbb{B}=\{0,n+1\}$. For the constant CNNs on ${\mathbb{Z}}_{n\times 1}$, the following three types of boundary conditions are considered:

• (i) (7) _n -N: constant CNNs with Neumann boundary condition on ${\mathbb{Z}}_{n\times 1}$;

• (ii) (7) _n -P: constant CNNs with periodic boundary condition on ${\mathbb{Z}}_{n\times 1}$;

• (iii) (7) _n -D: constant CNNs with Dirichlet boundary condition on ${\mathbb{Z}}_{n\times 1}$.

These boundary conditions are discrete analogues of the ones in PDEs; to be specific, a pattern $y_1\cdots y_n$ satisfies: (i) the Neumann boundary condition if $y_{n+1}=y_n$ and $y_0=y_1$; (ii) the periodic boundary condition if $y_{n+1}=y_1$; (iii) the Dirichlet boundary condition if $y_0$ and $y_{n+1}$ are prescribed.

Since $\mathbf{Y}\cong {\prod }_{j\in \mathrm{\Lambda }}{\mathbf{Y}}_j$, the total number of patterns of finite length in a constant CNN relates to the number of patterns in the subspaces. For each $j\in \mathrm{\Lambda }$, there is a transition matrix $T_j$ that is implemented for the investigation of the subspace ${\mathbf{Y}}_j$^a (cf.[17] and Section 4). Lemma 2.11 elucidates the exact number of mosaic patterns of length n of a constant CNN without the influence of the boundary condition. The verification is straightforward and is omitted.

Lemma 2.11 For$n\in \mathbb{N}$, write$n=k\ell +r$for some$k\in \mathbb{Z}$and$r=0,\dots ,\ell -1$. Then

$${\mathrm{\Gamma }}_n(\mathbf{Y})=(\prod _{j\le r}{\mathrm{\Gamma }}_{k+1}({\mathbf{Y}}_j))(\prod _{j>r}{\mathrm{\Gamma }}_k({\mathbf{Y}}_j)),$$

where$j\in \mathrm{\Lambda }$, and${\mathrm{\Gamma }}_q(X)$denotes the number of patterns of length q in X.

Let ${\mathrm{\Sigma }}_n^B(\mathbf{Y})$ denote the collection of output patterns of length n with boundary condition B, where $B=P,N$, and D stands for the periodic, Neumann, and Dirichlet boundary conditions, respectively. To find the exact number ${\mathrm{\Gamma }}_n^B(\mathbf{Y})=|{\mathrm{\Sigma }}_n^B(\mathbf{Y})|$, we introduce the following boundary matrices.

• (i) Periodic boundary matrix $R^P={\mathbf{1}}_{2^{2d}}^t\otimes (\left(\begin{array}{c}{E}^{(o)}\\ E^{(e)}\end{array}\right)\otimes {\mathbf{1}}_{2^{2d-1}})$. More precisely,

  $$R^P=\left(\begin{array}{ccc}{E}^{(o)}& \cdots & E^{(o)}\\ ⋮& ⋮\\ E^{(o)}& \cdots & E^{(o)}\\ E^{(e)}& \cdots & E^{(e)}\\ ⋮& ⋮\\ E^{(e)}& \cdots & E^{(e)}\end{array}\right)\in {\mathcal{M}}_{2^{4d}\times 2^{4d}}(\mathbb{R}).$$

• (ii) Dirichlet boundary matrices $L^{D_{-}}=I^{(u)}$, $L^{D_{+}}=I^{(l)}$, $R^{D_{-}}=I^{(o)}$, and $R^{D_{+}}=I^{(e)}$ stands for the left/right Dirichlet boundary condition that is given by ‘−’ and ‘+’, respectively.

• (iii) Neumann boundary matrices $L^N={\mathbf{1}}_{2^{2d}}^t\otimes (\left(\begin{array}{c}{E}^{(u)}\\ E^{(l)}\end{array}\right)\otimes {\mathbf{1}}_{2^{2d-1}})$, $R^N={\mathbf{1}}_{2^{2d-1}}^t\otimes ((E^{(o)}{E}^{(e)})\otimes {\mathbf{1}}_{2^{2d}})$. More precisely,

  $$L^N=\left(\begin{array}{ccc}{E}^{(u)}& \cdots & E^{(u)}\\ ⋮& ⋮\\ E^{(u)}& \cdots & E^{(u)}\\ E^{(l)}& \cdots & E^{(l)}\\ ⋮& ⋮\\ E^{(l)}& \cdots & E^{(l)}\end{array}\right),R^N=\left(\begin{array}{ccccc}{E}^{(o)}& E^{(e)}& \cdots & E^{(o)}& E^{(e)}\\ ⋮& ⋮& ⋮& ⋮\\ E^{(o)}& E^{(e)}& \cdots & E^{(o)}& E^{(e)}\end{array}\right).$$

Here ⊗ is the Kronecker product, E is a $2^{2d}\times 2^{2d}$ matrix with entries being 1’s, I is the $2^{2d}\times 2^{2d}$ identity matrix, and ${\mathbf{1}}_k$ is a $k\times 1$ column vector with entries being 1’s. Suppose that M is a $2k\times 2k$ matrix. Define $M^{(o)}/M^{(e)}$ by letting all the even/odd columns be zero vectors. Furthermore, $M^{(u)}/M^{(l)}$ indicates the matrix obtained from M by setting each of the lower-/upper-half rows as a zero vector.

Recall that a set function $\chi :2^{\mathbb{R}}\to {\{0,1\}}^{\mathbb{R}}$ is defined by $\chi (E)(x):={\chi }_E(x)=1$ if and only if $x\in E$ for E being a nonempty subset of ℝ. For $n\in \mathbb{N}$, define

$$k(n)=(\left[\frac{n}{\ell }\right]-2d){\chi }_{\mathbb{N}}(\left[\frac{n}{\ell }\right]-2d).$$

(14)

It is seen that $k(n)$ is a nonnegative integer. To clarify the formulae of the exact number of patterns of length n of constant CNNs with boundary conditions, we introduce some notations first. Suppose $n=k\ell +r$, where $0\le r\le \ell -1$. For $j\in \mathrm{\Lambda }$, set

$$K_j(n)=\{\begin{array}{ll}k(n),& j>r;\\ k(n)+1,& \text{otherwise}\end{array}$$

(15)

and

$$m_j(n)=\{\begin{array}{ll}\parallel T_j^{K_j(n)}\parallel ,& K_j(n)>0;\\ 2^k,& K_j(n)=0\text{ and }j>r;\\ 2^{k+1},& \text{otherwise}.\end{array}$$

(16)

Herein $\parallel M\parallel$ refers to the 1-norm of the matrix M. Lemma 2.12 demonstrates the explicit formulae of the number of patterns of length n with boundary conditions.

Lemma 2.12 Let$n=k\ell +r$, where$0\le r\le \ell -1$. Suppose$k\ge 2d+1$, then the exact number${\mathrm{\Gamma }}_n^B(\mathbf{Y})$with boundary condition$B\in \{P,N,D_{-},D_{+}\}$are as follows:

• (i) The periodic boundary condition:

  $${\mathrm{\Gamma }}_n^P(\mathbf{Y})=\{\begin{array}{ll}\parallel (T_1^{K_1(n)}\otimes T_1)\circ R^P\parallel {\prod }_{j>1}{m}_j(n),& r=0;\\ \parallel (T_1^{K_1(n)}\otimes T_{r+1}^{K_{r+1}(n)+1})\circ R^P\parallel {\prod }_{j\ne 1,r+1}{m}_j(n),& \mathit{\text{otherwise}}.\end{array}$$

  (17)

• (ii) The Dirichlet boundary condition:

  $${\mathrm{\Gamma }}_n^B(\mathbf{Y})=\{\begin{array}{ll}\parallel L^B{T}_{\ell }^{K_{\ell }(n)+2}{R}^B\parallel {\prod }_{j<\ell }{m}_j(n),& r=\ell -1;\\ \parallel (L^B{T}_{\ell }^{K_{\ell }(n)+1})\otimes (T_{r+1}^{K_{r+1}(n)+1}{R}^B)\parallel {\prod }_{j\ne r+1,\ell }{m}_j(n),& \mathit{\text{otherwise}},\end{array}$$

  (18)

where$B=D_{-}/D_{+}$means the pattern on the boundary is ‘$-/+$’.

• (iii) The Neumann boundary condition:

  $${\mathrm{\Gamma }}_n^N(\mathbf{Y})=\{\begin{array}{ll}\parallel L^N\circ (T_1^{K_1(n)+1}\otimes T_{\ell }^{K_{\ell }(n)+1})\circ R^N\parallel {\prod }_{j\ne 1,\ell }{m}_j(n),& r=0;\\ \parallel [(L^N\circ (T_1^{K_1(n)+1}\otimes T_{\ell }^{K_{\ell }(n)+1}))\otimes T_2^{K_2(n)+1}]\\ \circ (E_2\otimes R^N)\parallel {\prod }_{j=3}^{\ell -1}{m}_j(n),& r=1;\\ \parallel [T_{\ell -1}^{K_{\ell -1}(n)+1}\otimes (L^N\circ (T_1^{K_1(n)+1}\otimes T_{\ell }^{K_{\ell }(n)+1}))]\\ \circ (E_2\otimes R^N)\parallel {\prod }_{j=2}^{\ell -2}{m}_j(n),& r=\ell -1\end{array}$$

  (19)

and

$$\begin{array}{rcl}{\mathrm{\Gamma }}_n^N(\mathbf{Y})& =& \parallel L^N\circ (T_1^{K_1(n)+1}\otimes T_{\ell }^{K_{\ell }(n)+1})\parallel \\ \cdot \parallel (T_r^{K_r(n)+1}\otimes T_{r+1}^{K_{r+1}(n)+1})\circ R^N\parallel \prod _{j\ne 1,r,r+1,\ell }{m}_j(n),\end{array}$$

(20)

otherwise.

Here$E_2$is a$2\times 2$matrix with entries being 1’s, and ∘ means the Hadamard product.

Proof We address the proof of ${\mathrm{\Gamma }}_n^P(\mathbf{Y})$, where the other cases can be verified in an analogous method.

Suppose that $r=0$. It is seen from Lemma 2.11 that

$${\mathrm{\Gamma }}_n^P(\mathbf{Y})={\mathrm{\Gamma }}_{k+1}^P({\mathbf{Y}}_1)\prod _{j=2}^{\ell }{\mathrm{\Gamma }}_k({\mathbf{Y}}_j).$$

At the same time, $k\ge 2d+1$ indicates that $K_j(n)>0$ for all j. A straightforward examination demonstrates that

$${\mathrm{\Gamma }}_{k+1}^P({\mathbf{Y}}_1)=\parallel (T_1^{K_1(n)}\otimes T_1)\circ R^P\parallel$$

and ${\mathrm{\Gamma }}_k({\mathbf{Y}}_j)=m_j(n)$ for $j=2,\dots ,\ell$. Therefore, we have

$${\mathrm{\Gamma }}_n^P(\mathbf{Y})=\parallel (T_1^{K_1(n)}\otimes T_1)\circ R^P\parallel \prod _{j>1}{m}_j(n).$$

If $r>0$, then $y_{n+1}=y_1$ and

$${\mathrm{\Gamma }}_n^P(\mathbf{Y})={\mathrm{\Gamma }}_{k+1}^P({\mathbf{Y}}_1,{\mathbf{Y}}_{r+1})\prod _{j\ne 1,r+1}{\mathrm{\Gamma }}_k({\mathbf{Y}}_j),$$

where ${\mathrm{\Gamma }}_{k+1}^P({\mathbf{Y}}_1,{\mathbf{Y}}_{r+1})$ refers to the number of patterns

$$y_{1;1}\cdots y_{1;k+1}{y}_{r+1;1}\cdots y_{r+1;k+1}$$

with $y_{r+1;k+1}=y_{1;1}$, and $y_{1;1}\cdots y_{1;k+1}$ and $y_{r+1;1}\cdots y_{r+1;k+1}$ are patterns of length $k+1$ in ${\mathbf{Y}}_1$ and ${\mathbf{Y}}_{r+1}$, respectively. It is verified that

$${\mathrm{\Gamma }}_{k+1}^P({\mathbf{Y}}_1,{\mathbf{Y}}_{r+1})=\parallel (T_1^{K_1(n)}\otimes T_{r+1}^{K_{r+1}(n)+1})\circ R^P\parallel .$$

This derives

$${\mathrm{\Gamma }}_n^P(\mathbf{Y})=\parallel (T_1^{K_1(n)}\otimes T_{r+1}^{K_{r+1}(n)+1})\circ R^P\parallel \prod _{j\ne 1,r+1}{m}_j(n)$$

and completes the proof. □

Next, to study the influence of boundary conditions on the exact number of patterns of finite length, we consider the effect on the growth rate of the number of patterns; more specifically, the topological entropy of the output space Y. The topological entropy $h(X)$ of a space X is defined by

$$h(X)=\underset{n\to \mathrm{\infty }}{lim}\frac{log{\mathrm{\Gamma }}_n(X)}{n}.$$

(21)

The existence of $h(\mathbf{Y})$ comes immediately from the submultiplicativity of ${\{{\mathrm{\Gamma }}_n(\mathbf{Y})\}}_{n\in \mathbb{N}}$, which can be verified by applying Lemma 2.11. Theorem 2.13 declares the formula of the topological entropies of the constant CNNs, and the relation between the topological entropies of the constant CNNs and the classical CNNs.

Theorem 2.13$h(\mathbf{Y})=\frac{1}{\ell }{\sum }_{j\in \mathrm{\Lambda }}h({\mathbf{Y}}_j)$. Moreover, $h_B(\mathbf{Y})=h(\mathbf{Y})$for$B\in \{P,N,D_{-},D_{+}\}$provided${\mathbf{Y}}_j$is mixing for all$j\in \mathrm{\Lambda }$.

Proof For $n\in \mathbb{N}$, there exists a unique $n_{\ell }\in \mathbb{Z}$ such that

$$n_{\ell }\le \frac{n}{\ell }<n_{\ell }+1.$$

Lemma 2.11 infers that

$$\prod _{j=1}^{\ell }{\mathrm{\Gamma }}_{n_{\ell }}({\mathbf{Y}}_j)\le {\mathrm{\Gamma }}_n(\mathbf{Y})<\prod _j^{\ell }{\mathrm{\Gamma }}_{n_{\ell }+1}({\mathbf{Y}}_j).$$

Applying the squeeze theorem, we have

$$h(\mathbf{Y})=\underset{n\to \mathrm{\infty }}{lim}\frac{log{\mathrm{\Gamma }}_n}{n}=\frac{1}{\ell }\sum _{j=1}^{\ell }h({\mathbf{Y}}_j)=\frac{1}{\ell }\sum _{j=1}^{\ell }log\rho (T_j).$$

This completes the first part of the proof.

To evaluate the boundary effect on the topological entropy of Y, we demonstrate that $h_P(\mathbf{Y})=h(\mathbf{Y})$. The other cases can be done analogously. Let τ denote the smallest integer such that $T_j^{\tau }>0$ for $j\in \mathrm{\Lambda }$, restated, $T_j^{\tau }(p,q)>0$ for $1\le p,q\le 2^{2d}$. According to the definition of $R^P$,

$$\parallel (A\otimes B)\circ R^P\parallel =\sum _{i=l^{\prime }}A(i,i^{\prime })B(l,l^{\prime }).$$

Suppose $n=k\ell +r$. Lemma 2.12 implements

$$\begin{array}{rl}{\mathrm{\Gamma }}_{n+\tau \ell }^P(\mathbf{Y})& =\parallel (T_1^{K_1(n)+\tau }\otimes T_1)\circ R^P\parallel \prod _{j>1}{m}_j(n+\tau \ell )\\ =\left(\sum _{i=l^{\prime }}{T}_1^{K_1(n)+\tau }(i,i^{\prime })T_1(l,l^{\prime })\right)(\prod _{j>1}{m}_j(n+\tau \ell ))\\ \ge \prod _{j\ge 1}{m}_j(n)={\mathrm{\Gamma }}_n(\mathbf{Y}),\end{array}$$

if $r=0$, and

$$\begin{array}{rl}{\mathrm{\Gamma }}_{n+\tau \ell }^P(\mathbf{Y})& =\parallel (T_1^{K_1(n)+\tau }\otimes T_{r+1}^{K_{r+1}(n)+\tau +1})\circ R^P\parallel \prod _{j\ne 1,r+1}{m}_j(n+\tau \ell )\\ =\left(\sum _{i=l^{\prime }}{T}_1^{K_1(n)+\tau }(i,i^{\prime })T_{r+1}^{K_{r+1}(n)+\tau +1}(l,l^{\prime })\right)(\prod _{j\ne 1,r+1}{m}_j(n+\tau \ell ))\\ \ge \prod _{j\ge 1}{m}_j(n)={\mathrm{\Gamma }}_n(\mathbf{Y}),\end{array}$$

otherwise. On the other hand, it is easily checked that

$${\mathrm{\Gamma }}_n^P(\mathbf{Y})\le {\mathrm{\Gamma }}_n(\mathbf{Y}).$$

The above observation derives that

$${\mathrm{\Gamma }}_n^P(\mathbf{Y})\le {\mathrm{\Gamma }}_n(\mathbf{Y})\le {\mathrm{\Gamma }}_{n+\tau \ell }^P(\mathbf{Y}),$$

and thus we have $h_B(\mathbf{Y})=h(\mathbf{Y})$. □

The following theorem comes immediately from Theorem 2.13, the proof is omitted.

Theorem 2.14 The set of topological entropies of the constant CNNs is dense in the closed interval$[0,log2]$. More precisely, given$ϵ>0$and$\lambda \in [0,log2]$, there exists a constant CNN such that$|h(\mathbf{Y})-\lambda |<ϵ$.

3 Arithmetic cellular neural networks

This section elucidates another kind of inhomogeneous CNN-based LDS named arithmetic-type multiple cellular neural network (arithmetic CNN). It is seen that the templates of a constant CNN are periodic; in other words, the number of distinct templates is finite. This section investigates inhomogeneous CNNs whose number of distinct templates is infinite. First we consider a one-dimensional LDS with nearest neighborhood to interpret the idea of our methodology, then the derived results are generalized to general cases in the rest of this section.

3.1 Arithmetic cellular neural networks with nearest neighborhood

To clarify the study of an inhomogeneous LDS with nearest neighborhood, we consider the following system,

$$\frac{d}{dt}{x}_{q{2}^i}(t)=-x_{q{2}^i}(t)+z_q+a_{0;q}f(x_{q{2}^i}(t))+a_{1;q}f(x_{q{2}^{i+1}}(t)),$$

(22)

where $i\ge 0$ and $q\ge 1$ is odd. The feedback template $\mathbb{T}=[{\mathbb{T}}_1,{\mathbb{T}}_3,{\mathbb{T}}_5,\dots ]$ consists of infinitely many subtemplates ${\mathbb{T}}_q=[a_{0;q},a_{1;q}]$ for $q=2k+1$, $k\ge 0$, and the threshold $\mathbf{z}=[z_1,z_3,z_5,\dots ]$ is an infinite vector. An inhomogeneous CNN realized as (22) is called the arithmetic CNN.

Similar to the discussion in the previous subsection, we demonstrate that arithmetic CNNs are completely stable. Let ${\mathbf{x}}_q=(x_q,x_{2q},\dots ,x_{2^{n_q-1}q})$ denote the collection of cells related to initial coordinate q. Express (22) as

$${\dot{\mathbf{x}}}_q=-{\mathbf{x}}_q+{\mathbf{A}}_{q}F({\mathbf{x}}_q)+{\beta }_q,$$

(23)

where ${\mathbf{A}}_q$ and ${\beta }_q$ are similar to those defined in the previous subsection. A sufficient condition for the complete stability of (22) is presented as Theorem 3.1, which is a special case of Theorem 3.4.

Theorem 3.1 An arithmetic CNN is completely stable if${\mathbf{A}}_q(i,i)>1$and$K_q^{-1}(i,j)\ge 0$for all q, i, j, where

$$K_q(i,j)=\{\begin{array}{ll}{\mathbf{A}}_q(i,i)-1,& i=j;\\ -|{\mathbf{A}}_q(i,j)|,& \mathit{\text{otherwise}}.\end{array}$$

Following the complete stability of an arithmetic CNN is the spatial complexity of the output space and the influence of boundary conditions. Note that the output space $\mathbf{Y}=\{(y_i):y_i=f(x_i),i\in \mathbb{N}\}$ is different from the one in the previous subsection. The ordering matrix is then defined as

After redefining the ordering matrix, we obtain a sequence of transition matrices $T_q$ corresponding to ${\mathbf{Y}}_q=\{(y_{q{2}^{i-1}}):(y_i)\in \mathbf{Y}\}$ for $q=2k+1$, $k\ge 0$. The following theorem exhibits the computation of ${\mathrm{\Gamma }}_n(\mathbf{Y})$ and $h(\mathbf{Y})$. Furthermore, Theorem 3.2 is generalized to Theorem 3.7 for general arithmetic CNNs.

Theorem 3.2 Suppose that Y is the output space of an arithmetic CNN. Then

$${\mathrm{\Gamma }}_n(\mathbf{Y})=\left(\prod _{2q\le n}\parallel T_q^{[{log}_2\frac{n}{q}]-1}\parallel \right)\cdot \prod _{2q>n}2,$$

where q is odd and$[\cdot ]$is the Gauss function. Furthermore, the topological entropy of Y is

$$h(\mathbf{Y})=\underset{n\to \mathrm{\infty }}{lim}\frac{1}{n}(\sum _{2q\le n}log\parallel T_q^{[{log}_2\frac{n}{q}]-1}\parallel +\sum _{2q>n}log2).$$

For the influence of the boundary conditions, we define the boundary matrices as follows. Let

$$E^{(o)}=\left(\begin{array}{cc}1& 0\\ 1& 0\end{array}\right),E^{(e)}=\left(\begin{array}{cc}0& 1\\ 0& 1\end{array}\right),E^{(u)}=\left(\begin{array}{cc}1& 1\\ 0& 0\end{array}\right),E^{(l)}=\left(\begin{array}{cc}0& 0\\ 1& 1\end{array}\right).$$

The periodic boundary matrix $R^P$ is a $4\times 4$ matrix defined by

$$R^P=\left(\begin{array}{cc}{E}^{(o)}& E^{(o)}\\ E^{(e)}& E^{(e)}\end{array}\right),$$

and the left and right Neumann boundary matrices are then defined by

$$L^N=\left(\begin{array}{cc}{E}^{(u)}& E^{(u)}\\ E^{(l)}& E^{(l)}\end{array}\right)\text{and}{R}^N=\left(\begin{array}{cc}{E}^{(o)}& E^{(e)}\\ E^{(o)}& E^{(e)}\end{array}\right),$$

respectively. Furthermore, the Dirichlet boundary matrices are

$$L^{D_{-}}=R^{D_{-}}=\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right),L^{D_{+}}=R^{D_{+}}=\left(\begin{array}{cc}0& 0\\ 0& 1\end{array}\right).$$

To simplify the formulae of ${\mathrm{\Gamma }}_n^B(\mathbf{Y})$, the following theorem presents the specific case. The general case is postponed to Lemma 3.6 and Theorem 3.7.

Theorem 3.3 Suppose$n=2^k-1$for some k and Y is the output space of an arithmetic CNN. Then$h_B(\mathbf{Y})=h(\mathbf{Y})$, $B=P,N,D_{-},D_{+}$, if$T_q$are primitive for all q. Furthermore, the exact number of patterns of length n with boundary condition$B=P,N,D_{-},D_{+}$are as follows:

• The periodic boundary condition:

  $${\mathrm{\Gamma }}_n^P(\mathbf{Y})=\parallel (T_1^k\otimes T_1)\circ R^P\parallel \left(\prod _{q>1,2q\le n}\parallel T_q^{[{log}_2\frac{n}{q}]-1}\parallel \right)\cdot \prod _{2q>n}2.$$

  (24)

• The Neumann boundary condition:

  $${\mathrm{\Gamma }}_n^N(\mathbf{Y})=\parallel (T_1^k\otimes T_1)\circ R^N\parallel \left(\prod _{q>1,2q\le n}\parallel T_q^{[{log}_2\frac{n}{q}]-1}\parallel \right)\cdot \prod _{2q>n}2.$$

  (25)

• The Dirichlet boundary condition:

  $${\mathrm{\Gamma }}_n^P(\mathbf{Y})=\parallel T_1^{k+1}{R}^B\parallel \left(\prod _{q>1,2q\le n}\parallel T_q^{[{log}_2\frac{n}{q}]-1}\parallel \right)\cdot \prod _{2q>n}2,$$

  (26)

where$B=D_{-},D_{+}$.

3.2 Stability of arithmetic cellular neural networks

The rest of this section considers the inhomogeneous CNN-based LDS with the neighborhood consisting of infinitely many elements. A D-dimensional inhomogeneous CNN-based LDS is called an arithmetic CNN if the neighborhood , the template , and the threshold z are periodic up to a multiplication. More precisely, there exists a positive integer $\ell \ge 2$ such that^b

$${\mathcal{N}}_{\mathbf{i}}^{\prime }={\mathcal{N}}_{\mathbf{j}}^{\prime },{\mathbb{T}}_{\mathbf{i}}^{\prime }={\mathbb{T}}_{\mathbf{j}}^{\prime },z_{\mathbf{i}}=z_{\mathbf{j}}\text{whenever }\frac{j_k}{i_k}=0(mod\ell ),k=1,\dots ,D.$$

Herein

$${\mathbf{K}}_{\mathbf{i}}^{\prime }=\frac{{\mathbf{K}}_{\mathbf{i}}}{\mathbf{i}}=\{(\frac{j_1}{i_1},\dots ,\frac{j_D}{i_D}):\mathbf{j}\in {\mathbf{K}}_{\mathbf{i}}\}.$$

The essential description of a one-dimensional arithmetic CNN is that $\mathbf{z}={[z_j]}_{\ell \nmid j}$ and $\mathcal{N}={\{{\mathcal{N}}_j\}}_{\ell \nmid j}$. More precisely, a one-dimensional arithmetic CNN is realized as the form

$$\frac{d}{dt}{x}_i(t)=-x_i(t)+z_{\overline{i}}+\sum _{k\in {\mathcal{N}}_{\overline{i}}}{a}_{k;\overline{i}}f(x_{i\cdot {\ell }^k}(t)),i\in \mathbb{N},$$

(27)

where $\ell \nmid \overline{i}$, $i\ge \overline{i}$, , and ${\mathcal{N}}_{\overline{i}}=\{0,1,\dots ,d_{\overline{i}}\}$ for some $d_{\overline{i}}\in \mathbb{N}$.

Let $\mathrm{\Lambda }=\{j:\ell \nmid j\}$ be an infinite index set. The set of positive integers ℕ is then decomposed into the disjoint union of infinitely many subsets by

$$\mathbb{N}=\bigcup _{j\in \mathrm{\Lambda }}{\mathbb{N}}_j=\bigcup _{j\in \mathrm{\Lambda }}\{j{\ell }^i:i\ge 0\}:=\bigcup _{j\in \mathrm{\Lambda }}{\{j_i\}}_{i\ge 0},$$

where $j_i=j{\ell }^i$ for $j\in \mathrm{\Lambda }$, $i\ge 0$. Equation (27) can then be represented as

$$\frac{d}{dt}{x}_{j_i}(t)=-x_{j_i}(t)+z_j+\sum _{0\le k\le d_j}{a}_{k;j}f(x_{j_{i+k}}(t)),j\in \mathrm{\Lambda },i\ge 0.$$

(28)

In this case, the feedback template $\mathbb{T}={[{\mathbb{T}}_j]}_{j\in \mathrm{\Lambda }}$ consists of infinitely many smaller templates ${\mathbb{T}}_j=[a_{0;j},\dots ,a_{d_j;j}]$, and the threshold is $\mathbf{z}={[z_j]}_{j\in \mathrm{\Lambda }}$. Similar to the discussion in the previous section, Theorem 3.4 asserts that an arithmetic CNN is completely stable. The proof is omitted.

Theorem 3.4 Suppose that an arithmetic CNN is presented as

$${\dot{\mathbf{x}}}_j=-{\mathbf{x}}_j+{\mathbf{A}}_{j}F({\mathbf{x}}_j)+{\beta }_j,j\in \mathrm{\Lambda }.$$

Then the system is completely stable if$K_j\ge 0$for all$j\in \mathrm{\Lambda }$, where$K_j$comes from${\mathbf{A}}_j$defined in (10).

Suppose that y is a mosaic pattern; for each $j\in \mathrm{\Lambda }$ and $i\ge 0$, the necessary and sufficient condition for $y_{j_i}=1$ is

$$a_{0;j}-1+z_j>-\sum _{0<k\le d_j}{a}_{k;j}{y}_{j_{i+k}},$$

(29)

and the necessary and sufficient condition for $y_{j_i}=-1$ is

$$a_{0;j}-1-z_j>\sum _{0<k\le d_j}{a}_{k;j}{y}_{j_{i+k}}.$$

(30)

Set

$$\begin{array}{rl}{\mathcal{B}}_j(+)& =\{\boxed{y_0\cdots y_{d_j}}:y_0,\dots ,y_{d_j}\in \{-1,1\}\text{ satisfy (29)},y_0=1\},\\ {\mathcal{B}}_j(-)& =\{\boxed{y_0\cdots y_{d_j}}:y_0,\dots ,y_{d_j}\in \{-1,1\}\text{ satisfy (30)},y_0=-1\}.\end{array}$$

The set of admissible local patterns ℬ of an arithmetic CNN is then

$$\mathcal{B}(\mathbb{T},\mathbf{z})={({\mathcal{B}}_j(+),{\mathcal{B}}_j(-))}_{j\in \mathrm{\Lambda }}.$$

The output space Y is then represented as

$$\mathbf{Y}=\{y=(y_{j_i}):y_{j_i}\cdots y_{j_{i+d_j}}\in ({\mathcal{B}}_j(+),{\mathcal{B}}_j(-))\text{ for }j\in \mathrm{\Lambda },i\ge 0\}.$$

Recall that the output space Y of a constant CNN can be decomposed into finitely many subspaces ${\mathbf{Y}}_j$ such that ${\mathbf{Y}}_j$ is a SFT for each j. In other words, the output space of a constant CNN extends the concept of SFTs. The output space of an arithmetic CNN is decomposed into countable subspaces; more precisely, $\mathbf{Y}\cong {\prod }_{j\in \mathrm{\Lambda }}{\mathbf{Y}}_j$, where ${\mathbf{Y}}_j$ is determined by the basic set of admissible local patterns ${\mathcal{B}}_j=({\mathcal{B}}_j(+),{\mathcal{B}}_j(-))$. Theorem 3.5 demonstrates that the output space of an arithmetic CNN is a generalization of the so-called multiplicative shifts.

In [22], the authors introduced the concept of multiplicative subshifts in the context of symbolic dynamical systems. Let Ω be a subshift of ${\mathrm{\Sigma }}_m={\{0,\dots ,m-1\}}^{\mathbb{N}}$. Define

$$X_{\mathrm{\Omega }}=\{\omega ={({\omega }_k)}_{k=1}^{\mathrm{\infty }}:{({\omega }_{j{q}^i})}_{i\ge 0}\in \mathrm{\Omega }\text{ for }j\nmid i\},$$

which is invariant under the action of multiplicative integers:

$$({\omega }_k)\in X_{\mathrm{\Omega }}\Rightarrow {({\omega }_{rk})}_{k\in \mathbb{N}}\in X_{\mathrm{\Omega }}\text{for }r\in \mathbb{N}.$$

Then $X_{\mathrm{\Omega }}$ is called a multiplicative subshift. We define a semigroup action on ${\mathrm{\Sigma }}_m$ by the following. For any $r\in \mathbb{N}$ and $\omega =({\omega }_1,{\omega }_2,{\omega }_3,\dots )\in {\mathrm{\Sigma }}_m$, the action $r:{\mathrm{\Sigma }}_m\to {\mathrm{\Sigma }}_m$ is given by $r(\omega )=({\omega }_r,{\omega }_{2r},{\omega }_{3r},\dots )$. It is seen that $X_{\mathrm{\Omega }}$ is invariant under the action. In other words, $(X_{\mathrm{\Omega }},\mathbb{N})$ defines a multiplicative subshift.

A straightforward examination indicates that the output space Y of an arithmetic CNN is a multiplicative subshift if the neighborhood and the templates of (27) are invariant; restated, ${\mathcal{N}}_i={\mathcal{N}}_j$ and ${\mathbb{T}}_i={\mathbb{T}}_j$ for all $i,j\in \mathrm{\Lambda }$. The proof is omitted.

Theorem 3.5 Given a set of templates$(\mathbb{T},\mathbf{z})$. Let Y be the solution space of the arithmetic CNN with respect to$(\mathbb{T},\mathbf{z})$. Then Y is a multiplicative subshift if${\mathcal{N}}_i={\mathcal{N}}_j$and${\mathbb{T}}_i={\mathbb{T}}_j$for all$i,j\in \mathrm{\Lambda }$. More precisely,

$$\mathbf{Y}\cong X_{\mathrm{\Omega }},$$

where Ω is the SFT that comes from the output space of the classical CNN with respect to the template$({\mathbb{T}}_1,z_1)$.

3.3 Boundary effects on arithmetic cellular neural networks

Recall that a set function $\chi :2^{\mathbb{R}}\to {\{0,1\}}^{\mathbb{R}}$ is defined by $\chi (E)(x):={\chi }_E(x)=1$ if and only if $x\in E$ for E is a nonempty subset of ℝ. For $n\in \mathbb{N}$ and $j\in \mathrm{\Lambda }$ such that $j<n$, define

$$k_j(n)=\left[{log}_{\ell }\frac{n}{j}\right],$$

(31)

$$K_j(n)=(k_j(n)-d_j+1){\chi }_{\mathbb{N}}(k_j(n)-d_j+1).$$

(32)

It is seen that both $k_j(n)$ and $K_j(n)$ are nonnegative integers. To clarify the formulae of the exact number of patterns of length n of an arithmetic CNN with boundary condition, we introduce some notations first. Set

$$m_j(n)=\{\begin{array}{ll}\parallel T_j^{K_j(n)}\parallel ,& K_j(n)>0;\\ 2^{k_j(n)+1},& \text{otherwise}.\end{array}$$

(33)

Recall that $T_j$ is the transition matrix of ${\mathbf{Y}}_j$.

The exact number of patterns of the arithmetic CNNs with boundary condition is obtained via a small modification of the discussion in the proof of Lemma 2.12. Before presenting the formulae, we assume that $d_j=d>1$ for all j and redefine the boundary matrices as follows. Suppose that E is a $2^{d-1}\times 2^{d-1}$ matrix with all entries being 1’s. The periodic boundary matrix, left and right Neumann boundary matrices are $2^{2d-2}\times 2^{2d-2}$ matrices given by

$$R^P=\left(\begin{array}{ccc}{E}^{(o)}& \cdots & E^{(o)}\\ ⋮& ⋮\\ E^{(o)}& \cdots & E^{(o)}\\ E^{(e)}& \cdots & E^{(e)}\\ ⋮& ⋮\\ E^{(e)}& \cdots & E^{(e)}\end{array}\right),L^N=\left(\begin{array}{ccc}{E}^{(u)}& \cdots & E^{(u)}\\ ⋮& ⋮\\ E^{(u)}& \cdots & E^{(u)}\\ E^{(l)}& \cdots & E^{(l)}\\ ⋮& ⋮\\ E^{(l)}& \cdots & E^{(l)}\end{array}\right)$$

and

$$R^N=\left(\begin{array}{ccccc}{E}^{(o)}& E^{(e)}& \cdots & E^{(o)}& E^{(e)}\\ ⋮& ⋮& ⋮& ⋮\\ E^{(o)}& E^{(e)}& \cdots & E^{(o)}& E^{(e)}\end{array}\right),$$

respectively. The left and right Dirichlet boundary matrices are defined as

$$L^{D_{-}}=I^{(u)},L^{D_{+}}=I^{(l)},R^{D_{-}}=I^{(o)},R^{D_{+}}=I^{(e)},$$

where I is the $2^{d-1}\times 2^{d-1}$ identity matrix. The formulae of ${\mathrm{\Gamma }}_n^B(\mathbf{Y})$ are addressed as follows and the demonstration is omitted.

Lemma 3.6 Suppose$n={\ell }^k-1$, then

$${\mathrm{\Gamma }}_n^P(\mathbf{Y})=\parallel (T_1^{K_1(n)}\otimes T_1)\circ R^P\parallel \prod _{j\ge 2}{m}_j(n),$$

(34)

$${\mathrm{\Gamma }}_n^B(\mathbf{Y})=\parallel T_1^{K_1(n)+1}{R}^B\parallel \prod _{j\ge 2}{m}_j(n),B\in \{D_{-},D_{+}\},$$

(35)

$${\mathrm{\Gamma }}_n^N(\mathbf{Y})=\parallel (T_1^{K_1(n)}\otimes T_1)\circ R^N\parallel \prod _{1<j<n-1}{m}_j(n).$$

(36)

Suppose$n=q{\ell }^k-1$, where$\ell \nmid q$, $q\ne 1$, and$k\ge 0$. Then:

• (i) The periodic boundary condition:

  $${\mathrm{\Gamma }}_n^P(\mathbf{Y})=\parallel (T_1^{k_1(n)+1}\otimes T_q^{k_q(n)})\circ R^P\parallel \prod _{1<j\ne q}{m}_j(n).$$

  (37)

• (ii) The Dirichlet boundary condition:

  $${\mathrm{\Gamma }}_n^B(\mathbf{Y})=\parallel T_q^{k_q(n)+1}{R}^B\parallel \prod _{j\ne q}{m}_j(n),B\in \{D_{-},D_{+}\}.$$

  (38)

• (iii) The Neumann boundary condition:

  $${\mathrm{\Gamma }}_n^N(\mathbf{Y})=m_q(n+1)\prod _{j\ne q}{m}_j(n).$$

  (39)

Theorem 3.7 formulates the topological entropy of the output space of an arithmetic CNN with/without boundary conditions.

Theorem 3.7 Suppose that there exists$d\in \mathbb{N}$such that$d_j\le d$for$j\in \mathrm{\Lambda }$. Then

$$h(\mathbf{Y})=\underset{n\to \mathrm{\infty }}{lim}\frac{1}{n}\sum _{j\le n,j\in \mathrm{\Lambda }}log{m}_j(n),$$

(40)

where$m_j(n)$is defined in (33). Furthermore, $h_B(\mathbf{Y})=h(\mathbf{Y})$for$B\in \{P,N,D_{-},D_{+}\}$provided${\mathbf{Y}}_j$is mixing for$j\in \mathrm{\Lambda }$.

Proof The calculation of $h(\mathbf{Y})$ is presented; the effect of the boundary condition on the topological entropy can be elucidated via similar discussion, and as with the proof of Theorem 2.13, thus is omitted.

Observe that

$${\mathrm{\Gamma }}_n(\mathbf{Y})=\prod _{j\le n,j\in \mathrm{\Lambda }}{\mathrm{\Gamma }}_{K_j(n)}({\mathbf{Y}}_j)=\prod _{j\le n,j\in \mathrm{\Lambda }}{m}_j(n).$$

Hence we have

$$h(\mathbf{Y})=\underset{n\to \mathrm{\infty }}{lim}\frac{log{\mathrm{\Gamma }}_n(\mathbf{Y})}{n}=\underset{n\to \mathrm{\infty }}{lim}\frac{1}{n}\sum _{j\le n,j\in \mathrm{\Lambda }}log{m}_j(n).$$

This completes the proof. □

The numerical experiment asserts that, similar to Theorem 2.14, the set of topological entropies of the arithmetic CNNs is dense in the closed interval $[0,log2]$. The theoretical proof of the following conjecture is not complete yet.

Conjecture 3.8 Given$ϵ>0$and$\lambda \in [0,log2]$, there exists an arithmetic CNN such that$|h(\mathbf{Y})-\lambda |<ϵ$.

4 Examples

4.1 One-dimensional cellular neural networks

Example 4.1 Consider a constant CNN with templates $\mathbb{T}=[{\mathbb{T}}_1,{\mathbb{T}}_2]$ and z being given by

$${\mathbb{T}}_1=[-4,6,2],{\mathbb{T}}_2=[-2,3,-4]\text{and}\mathbf{z}=[-5,2].$$

(Notably, $d=1$ and $\ell =2$ in this case.) The transition matrices $T_1$, $T_2$ for ${\mathbf{Y}}_1$, ${\mathbf{Y}}_2$ are

$$T_1=\left(\begin{array}{cccc}1& 1& 0& 0\\ 0& 0& 1& 1\\ 1& 1& 0& 0\\ 0& 0& 0& 0\end{array}\right)\text{and}{T}_2=\left(\begin{array}{cccc}0& 1& 0& 0\\ 0& 0& 1& 1\\ 0& 1& 0& 0\\ 0& 0& 1& 0\end{array}\right),$$

respectively. Theorem 2.13 infers that

$$h(\mathbf{Y})=\frac{1}{2}(h({\mathbf{Y}}_1)+h({\mathbf{Y}}_2))=\frac{1}{2}(logg+log{\lambda }_1)\approx log1.4640,$$

where $g=(1+\sqrt{5})/2$ is the golden mean and ${\lambda }_1=1.3247\dots$ is the maximal root of $x^3-x-1=0$.

To estimate the exact number of the mosaic patterns of length n with boundary conditions, we consider the case where $n=20$. It follows that $K_1(20)=K_2(20)=8$. Let

$$\begin{array}{c}{E}^{(o)}=\left(\begin{array}{cccc}1& 0& 1& 0\\ 1& 0& 1& 0\\ 1& 0& 1& 0\\ 1& 0& 1& 0\end{array}\right),E^{(e)}=\left(\begin{array}{cccc}0& 1& 0& 1\\ 0& 1& 0& 1\\ 0& 1& 0& 1\\ 0& 1& 0& 1\end{array}\right),\hfill \\ E^{(u)}=\left(\begin{array}{cccc}1& 1& 1& 1\\ 1& 1& 1& 1\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right),E^{(l)}=\left(\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 1& 1& 1& 1\\ 1& 1& 1& 1\end{array}\right).\hfill \end{array}$$

The periodic and Neumann boundary matrices are then

$$R^P=\left(\begin{array}{cccc}{E}^{(o)}& E^{(o)}& E^{(o)}& E^{(o)}\\ E^{(o)}& E^{(o)}& E^{(o)}& E^{(o)}\\ E^{(e)}& E^{(e)}& E^{(e)}& E^{(e)}\\ E^{(e)}& E^{(e)}& E^{(e)}& E^{(e)}\end{array}\right)$$

and

$$L^N=\left(\begin{array}{cccc}{E}^{(u)}& E^{(u)}& E^{(u)}& E^{(u)}\\ E^{(u)}& E^{(u)}& E^{(u)}& E^{(u)}\\ E^{(l)}& E^{(l)}& E^{(l)}& E^{(l)}\\ E^{(l)}& E^{(l)}& E^{(l)}& E^{(l)}\end{array}\right),R^N=\left(\begin{array}{cccc}{E}^{(o)}& E^{(e)}& E^{(o)}& E^{(e)}\\ E^{(o)}& E^{(e)}& E^{(o)}& E^{(e)}\\ E^{(o)}& E^{(e)}& E^{(o)}& E^{(e)}\\ E^{(o)}& E^{(e)}& E^{(o)}& E^{(e)}\end{array}\right),$$

respectively. Then the exact number of the mosaic patterns of length 20 with periodic boundary condition is

$${\mathrm{\Gamma }}_{20}^P(\mathbf{Y})=\parallel (T_1^8\otimes T_1)\circ R^P\parallel \cdot \parallel T_2^8\parallel =19\text{,}758;$$

the exact number of the mosaic patterns of length 20 with Neumann boundary condition is

$${\mathrm{\Gamma }}_{20}^N(\mathbf{Y})=\parallel L^N\circ (T_1^9\otimes T_2^9)\circ R^N\parallel =3\text{,}647.$$

Furthermore, the Dirichlet boundary matrices are given by

$$\begin{array}{c}{L}^{D_{-}}=\left(\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right),R^{D_{-}}=\left(\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 0\end{array}\right),\hfill \\ L^{D_{+}}=\left(\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right),R^{D_{+}}=\left(\begin{array}{cccc}0& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 1\end{array}\right).\hfill \end{array}$$

The exact number of the mosaic patterns of length 20 enclosed by the pattern ‘−’ is

$${\mathrm{\Gamma }}_{20}^{D_{-}}(\mathbf{Y})=\parallel (L^{D_{-}}\cdot T_2^9)\otimes (T_1^9\cdot R^{D_{-}})\parallel =4\text{,}032;$$

the exact number of the mosaic patterns of length 20 enclosed by the pattern ‘+’ is

$${\mathrm{\Gamma }}_{20}^{D_{+}}(\mathbf{Y})=\parallel (L^{D_{+}}\cdot T_2^9)\otimes (T_1^9\cdot R^{D_{+}})\parallel =3\text{,}024.$$

Suppose that the template $\mathbb{T}=[{\mathbb{T}}_1,{\mathbb{T}}_2]$ is given by

$${\mathbb{T}}_1=[-4,a_1,2],{\mathbb{T}}_2=[-2,a_2,-4]\text{and}\mathbf{z}=[z_1,z_2],$$

where $a_1$, $a_2$, $z_1$, and $z_2$ are unknown. It is known that (cf.[17, 32]) there are only finite possibilities of topological entropies for ${\mathbf{Y}}_1$ and ${\mathbf{Y}}_2$ as the parameters vary. More precisely,

$$h({\mathbf{Y}}_1)\in \{log{\lambda }_1,logg,log{\lambda }_2,log2\},h({\mathbf{Y}}_2)\in \{logg,log2\},$$

where ${\lambda }_2=1.8393\dots$ is the maximal root of $x^3-x^2-x-1=0$. The topological entropies $h(\mathbf{Y})$ with the parameters varying are seen in Table 1.

Table 1 The topological entropy $\mathit{h}\mathbf{(}\mathbf{Y}\mathbf{)}$ of constant CNNs with 2-components and templates being given by $\mathbf{(}{\mathbb{T}}_{\mathbf{1}}\mathbf{,}{\mathit{z}}_{\mathbf{1}}\mathbf{)}\mathbf{=}\mathbf{(}\mathbf{[}\mathbf{-}\mathbf{4}\mathbf{,}{\mathit{a}}_{\mathbf{1}}\mathbf{,}\mathbf{2}\mathbf{]}\mathbf{,}{\mathit{z}}_{\mathbf{1}}\mathbf{)}$ and $\mathbf{(}{\mathbb{T}}_{\mathbf{2}}\mathbf{,}{\mathit{z}}_{\mathbf{2}}\mathbf{)}\mathbf{=}\mathbf{(}\mathbf{[}\mathbf{-}\mathbf{2}\mathbf{,}{\mathit{a}}_{\mathbf{2}}\mathbf{,}\mathbf{-}\mathbf{4}\mathbf{]}\mathbf{,}{\mathit{z}}_{\mathbf{2}}\mathbf{)}$

Example 4.2 Consider an arithmetic CNN with $\ell =2$ and an invariant template ${\mathbb{T}}_j=[a_0,a_1]$, $z_j=z$ for $j\in \mathit{odd}$. Suppose that the transition matrix of ${\mathbf{Y}}_j$ is

$$T_j\equiv T=\left(\begin{array}{cc}1& 1\\ 1& 0\end{array}\right)\text{for }j\in \mathit{odd}.$$

In other words, ${\mathbf{Y}}_j$ is a golden mean shift with topological entropy $h({\mathbf{Y}}_j)=logg$ for all j. We remark that Fan et al.[23] investigated the Minkowski dimension of Y. To compute the topological entropy of Y, for $n\in \mathbb{N}$, let

$$\begin{array}{c}{I}_0=\{1,3,\dots ,2n_0-1\},\hfill \\ I_1=\{2\cdot 1,2\cdot 3,\dots ,2\cdot (2n_1-1)\},\hfill \\ I_2=\{2^2\cdot 1,2^2\cdot 3,\dots ,2^2\cdot (2n_2-1)\},\hfill \\ ⋮\hfill \\ I_k=\{2^k\cdot 1\}\hfill \end{array}$$

be the sets of integers such that $m\in {\bigcup }_{j=0}^k{I}_j$ if and only if $m\le n$. A straightforward verification infers that

$$n_j=[\frac{n}{2^{j+1}}+\frac{1}{2}]$$

for $j<k$ and $k=[{log}_{2}n]$. In other words, $\{1,2,\dots ,n\}={\bigcup }_{j=0}^k{I}_j$ and ${\{n_j\}}_{j=0}^k$ is decreasing with $n_k=1$. Suppose that $\{a_n\}$ is the Fibonacci sequence defined by

$$a_0=1,a_1=2,a_n=a_{n-1}+a_{n-2}(n\ge 2).$$

By induction we derive that

$${\mathrm{\Gamma }}_n(\mathbf{Y})=a_1^{n_0-n_1}{a}_2^{n_1-n_2}\cdots a_{k-1}^{n_{k-2}-n_{k-1}}{a}_k^{n_{k-1}-n_k}{a}_{k+1}^{n_k}.$$

Therefore,

$$\begin{array}{rl}h(\mathbf{Y})& =\underset{n\to \mathrm{\infty }}{lim}\frac{1}{n}{\mathrm{\Gamma }}_n(\mathbf{Y})\\ =\underset{n\to \mathrm{\infty }}{lim}\frac{1}{n}(log{a}_{k+1}+\sum _{j=1}^{[{log}_{2}n]}([\frac{n}{2^j}+\frac{1}{2}]-[\frac{n}{2^{j+1}}+\frac{1}{2}])log{a}_j)\\ =\frac{1}{2}\sum _{n=1}^{\mathrm{\infty }}\frac{log{a}_n}{2^n}=0.248137\dots .\end{array}$$

4.2 Two-dimensional constant cellular neural networks

Let $\mathrm{\Lambda }=\{1,2\}\times \{1,2\}$. For $(k_1,k_2)\in \mathrm{\Lambda }$, set

$${\mathbb{Z}}_{(k_1,k_2)}^2=\{(m_1,m_2):m_1=2C_1+k_1,m_2=2C_2+k_2\text{ for some }{C}_1,C_2\in \mathbb{Z}\}.$$

The two-dimensional lattice ${\mathbb{Z}}^2$ is written as the union of non-overlapping subspaces ${\mathbb{Z}}^2={\bigcup }_{(k_1,k_2)\in \mathrm{\Lambda }}{\mathbb{Z}}_{(k_1,k_2)}^2$. For each $k_1$, $k_2$, we index the entries in ${\mathbb{Z}}_{k_1,k_2}^2$ as ${\mathbb{Z}}_{(k_1,k_2)}^2=\{({k_1}_i,{k_2}_j),i,j\in \mathbb{Z}\}$. Consider the two-dimensional constant CNNs of the form

$$\frac{d}{dt}{x}_{{k_1}_i,{k_2}_j}=-x_{{k_1}_i,{k_2}_j}+z_{{k_1}_i,{k_2}_j}+\sum _{|k|,|\ell |\le 1}{a}_{k,\ell ;k_1,k_2}f(x_{{k_1}_{i+k},{k_2}_{j+\ell }}),$$

(41)

with template $\mathbb{T}={[{\mathbb{T}}_{(k_1,k_2)}]}_{(k_1,k_2)\in \mathrm{\Lambda }}$ being given by

$${\mathbb{T}}_{(k_1,k_2)}=\left[\begin{array}{ccc}0& b_{k_1,k_2}& 0\\ b_{k_1,k_2}& a_{k_1,k_2}& b_{k_1,k_2}\\ 0& b_{k_1,k_2}& 0\end{array}\right].$$

(42)

Fix $(k_1,k_2)\in \mathrm{\Lambda }$, Juang and Lin [17] studied (41) systematically and estimated the lower bound of the topological entropy $h({\mathbf{Y}}_{k_1,k_2})$. More precisely, the lower bound of the topological entropy $h({\mathbf{Y}}_{k_1,k_2})$ is

$$\frac{log2}{16},\frac{log4}{12},\frac{log4}{9},\frac{log4}{4},\frac{log10}{4}\text{and}log2.$$

Suppose that the template $\mathbb{T}={[{\mathbb{T}}_{(k_1,k_2)}]}_{(k_1,k_2)\in \mathrm{\Lambda }}$ and $z={[z_{(k_1,k_2)}]}_{(k_1,k_2)\in \mathrm{\Lambda }}$ are chosen so that

$$h({\mathbf{Y}}_{1,1})\ge \frac{log4}{12},h({\mathbf{Y}}_{1,2})\ge \frac{log4}{4},h({\mathbf{Y}}_{2,1})\ge \frac{log2}{16}\text{and}h({\mathbf{Y}}_{2,2})=log2.$$

Then

$$h(\mathbf{Y})\ge \frac{1}{4}(\frac{log4}{12}+\frac{log4}{4}+\frac{log2}{16}+log2)=\frac{83}{192}log2.$$

A detailed and complete investigation is postponed to the upcoming manuscript.

5 Conclusions

The present paper studies two types of one-dimensional inhomogeneous CNN-based LDS, say, constant- and arithmetic-type multiple CNNs, which are a generalization of the classical CNNs. Sufficient conditions for the complete stability of constant and arithmetic CNNs are revealed. Since there is a wide range of parameters making the system completely stable, it is essential to investigate the complexity of mosaic patterns of the given system. A systematic methodology is proposed to interpret the exact number of mosaic patterns of inquired length and the topological entropy of the output space. Furthermore, the exact number of mosaic patterns and the topological entropy of the output space under the influence of three boundary conditions, say, the periodic, Neumann, and Dirichlet boundary conditions, are obtained. Remarkably, the boundary condition does not influence the topological entropy under some presumption.

The reveal that the set of topological entropies of the output spaces of constant CNNs is dense in the close interval $[0,log2]$ indicates how rich phenomena constant CNNs could exhibit. Although there is a lack of rigorous proof for the density of the set of topological entropies of arithmetic CNNs, numerical experiments assert an affirmative result.

The methodology we propose in this investigation can be applied to multi-dimensional cases. A detailed discussion is on-going.

Endnotes

^a Notably every subspace ${\mathbf{Y}}_j$ is a subshift of finite type (SFT) in symbolic dynamical systems, and thus can be studied via the graph theory and matrix theory. The reader is referred to [17] and [38] for more details.

^b An arithmetic CNN is a classical CNN for the case that $\ell =1$, this makes the requirement essential.