\( \def\<#1>{\left<#1\right>} \newcommand{\OR}{\mathrm{OR}} \newcommand{\XOR}{\mathrm{XOR}} \newcommand{\SUM}{\mathrm{SUM}} \newcommand{\GEN}{\mathrm{GEN}} \newcommand{\Cov}{\mathrm{Cov}} \newcommand{\blog}{\log_2 } \)

A lower bound for OR-circuits computing free matrices

Say that a boolean $n\times n$ matrix with $n=2^m$ is free if it has no $2^k\times 2^{m-k}$ rectangles (all-1 submatrices) for $k=0,1,\ldots,m$. Prominent examples of free matrices are Hadamard matrices (as a consequence of Lindsey's lemma).
Theorem (Sergeev [3]): Let $A$ be a free $n\times n$ matrix, and let $r_i$ be the number of ones in its $i$-th row. Then \[ \OR(A)\geq \frac{1}{n}\sum_{i=1}^n r_i\log r_i\geq \frac{|A|}{n}\log\frac{|A|}{n}. \]
For the proof, we need the following simple fact about directed acyclic graphs of outdegree at most $2$. We assume that there is only one source node (of zero in-degree). Nodes of zero out-degree are called leaves. The weight of a node is the number of leaves reachable from it. The weight of the entire graph is the sum of weights of its nodes. If $L(n)$ denotes the smallest possible weight of a graph with $n$ leaves, then we have \[ L(n)\geq n\log n\tag{*} \] Proof: It is enough to show this for the minimum $L(n)$ taken over all binary threes with $n$ leaves. We do this by induction on $n$. The root has two children. Let $n_0$ and $n_1$ be the numbers of leaves in their subtrees; hence, $n=n_0+n_1$. Then \[ L(n)=n + L(n_0) + L(n_1) \geq n +n_0\log n_0 + n_1\log n_1 \geq n+(n_0+n_1)\log \frac{n_0+n_1}{2} =n\log n. \] The second inequality here follows from Jensen's inequality, since the function $f(x)=x\log x$ is convex. $\Box$

Proof of Theorem: Take a minimal OR-circuit for $A$. Say that a node $u$ can "see" a node $v$ if there is a path from $v$ to $u$. Define the weight of a node as the number of input nodes seen by it. Let $V_i$ be the set of all nodes seen by the $i$-th output node. Thus, each node in $V_i$ has weight at most $r_i$. For an integer $0\leq d < m$, let $W_{d}$ be the set of all nodes whose weight lies in the interval $(n2^{-d}, n2^{-d+1}]$. Thus, $\sum_d |W_d|$ is the total number of gates. Our goal is to lower bound this sum.

By the freeness of our matrix, no node in $W_d$ can be seen by more than $2^d$ output nodes. This gives \[ \sum_{i=1}^n |V_i\cap W_d|\leq |W_d|2^d. \tag{1} \] Fix now one $i$, and consider the sum $S_i$ of weights of all nodes in $V_i$. By (*), we have that $S_i\geq r_i\log r_i$. On the other hand, since every node in $W_d$ has weight at most $n2^{-d+1}$, we have that \[ S_i\leq \sum_{d=0}^{m-1}n2^{-d+1}|V_i\cap W_d|. \] Thus, for every $i=1,\ldots,n$, \[ \sum_{d}2^{-d}|V_i\cap W_d| > \frac{1}{n}r_i\log r_i. \tag{2} \] Form (1) and (2), the desired lower bound follows: \[ \OR(A) = \sum_d |W_d|\geq \sum_d 2^{-d}\sum_{i}|V_i\cap W_d| =\sum_i \sum_d 2^{-d}|V_i\cap W_d| > \frac{1}{n}\sum_i r_i\log r_i. \] $\Box$

If we define the area of an $a\times b$ rectangle to be $ab$, and let $r(A)$ be the maximal area of a rectangle in $A$, then the same argument yields

Corollary: For every $n\times n$ boolean matrix $A$, \[ \OR(A)\geq \frac{|A|}{r(A)}\log\frac{|A|}{n}. \]
Proof: The only difference is that now every node in $W_d$ can be seen by at most $r(A)/n2^{-d}$ output nodes. Thus, (1) turns to \[ \sum_{i=1}^n |V_i\cap W_d|\leq |W_d|2^d\frac{r(A)}{n}. \tag{1} \] The rest is the same. $\Box$ For every $n\times n$ Hadamard matrix $H$, we have $|H|=\Omega(n^2)$ and $r(H)\leq n$. Thus, \[ \OR(H)=\Omega(n\log n). \]
S.J. May 2013