\( \def\<#1>{\left<#1\right>} \newcommand{\OR}[1]{\mathrm{OR}(#1)} \newcommand{\XOR}[1]{\mathrm{XOR}(#1)} \newcommand{\SUM}[1]{\mathrm{SUM}(#1)} \newcommand{\GEN}{\mathrm{GEN}} \newcommand{\Cov}{c} \newcommand{\Dec}{d} \newcommand{\tr}{\mathrm{tr}} \newcommand{\pr}{\mathrm{rk}_+} \newcommand{\br}{\mathrm{rk}_{\lor}} \newcommand{\Geq}{\succcurlyeq} \newcommand{\Leq}{\preccurlyeq} \newcommand\LL{\mathsf{L}} \)

Problem 7.4 is solved

Let $A$ be an $m\times n$ matrix with entries from $\{0,1,\ldots,q-1\}$. Recall that the sum-complexity $\LL(A)$ of a $A$ is the minimum number of addition operations required to compute the linear transformation $y=Ax$. Such a circuit for $A=(a_{i,j})$ has $n$ inputs, $m$ outputs, and the number of paths from the $j$-th input to the $i$-th output is exactly $a_{i,j}$. The corresponding Shannon function is $\LL_d(m,n)=\max\{\LL(A)\colon \mbox{$A$ is an $m\times n$ matrix}\}$, and is $\LL_d(m,n)$ when restricted to circuits of depth at most $d$.

As mentioned in the book (Theorem 2.4), tight asymptotic estimate when the matrices are boolean ($q=2$) and square matrices ($m=n$) was proved by Nechiporuk: \[ \LL(n,n)\sim\LL_3(n,n) \sim \frac{n^2}{2\log_2 n}\,. \] The situation with the case of non-square matrices (when $m \ll n$) as well as of matrices over non-boolean domains ($q=q(n)$ large) is more difficult.

Nechiporuk has proved tight asymptotics $\LL(m,n) \sim \LL_3(m,n)$ when $\ln m \sim a \ln n$ for some certain rational values $a$. Pippenger obtained tight asymptotics together with a rather accurate secondary term not just for arbitrary $m$, $n$ (as long as $m$ is not too small compared to $n$) but also for non-boolean domains $q>2$. He has made this by relaxing the depth constraint. So, the question about the asymptotic behaviour of $\LL(m,n)$ was actually closed (by now, the only case when asymptotic is not known is when $m = c \log n$, $c > 2-o(1)$ is a constant).

But the question on whether the asymptotics can be achieved in some constant depth (as conjectured by Nechiporuk) remained open.

Problem 7.4: Does $\LL_d(m,n) \sim mn/\log(mn)$ hold for all $\log n \ll m \leq n$ and a constant $d$?
Recently, Igor resolved this problem by a clever combination of some Pippenger's ideas.
Theorem (Sergeev [1]): If $m\leq n$ and $m=\Omega(\ln^{3/2}n)$, then \[ \LL_3(m,n) \leq (1+\tau)\frac{mn}{\log_2(mn)}\qquad \mbox{where}\qquad \tau = \Theta\left(\sqrt{\ln\ln n/\ln n}\right). \]
The theorem holds (with $\log_2$ replaced by $\log_q$) also for any $q\leq (\ln n)^{O(1)}$. For larger values of $q$ up to $o(n/\ln^2n)$, the same upper bound holds in depth $d=4$. For even larger values of $q$ up to $n^{O(1)}$, the upper bound holds for circuits of larger, but still constant depth.

Reference:

  1. I.S. Sergeev, Rectifier circuits of bounded depth, Journal of Applied and Industrial Mathematics 12(1) (2018) 153–166.

Posted April 15, 2017. Link to reference [1] added March 14, 2018.



S. Jukna