In this article computation and comparison of causality measures which are used in determination of brain connectivity patterns is investigated. Main analyzed examples included published computation and comparisons of Directed Transfer Function ‐ DTF and Partial Directed Coherence ‐ PDC. It proved that serious methodology mistakes were involved in measure computations and comparisons. It is shown that the neighborhood of zero is of accented importance in such evaluations and that the issues of semantic stability have to be treated with more attention. Published results on the relationship of these two important measures are partly unstable with small changes of zero threshold and pictures of involved brain structures deduced from the cited articles have to be corrected. Analysis of the operators involved in evaluation and comparisons is given with suggestions for their improvement and complementary additional actions.
Keywords:Causality measures; Brain connectivity patterns; Connectivity measure comparisons; Direct transfer function; Partial directed coherence
This article is intended to researchers involved in brain connectivity mapping or using them. Number of sensitive issues in computations of measure and measure comparisons are discussed, illustrating actual problems with the examples published in the articles of authors with highest authority in the field.
After success of Granger causality measure, the growing number of different causality measures have been developed aiming to model brain connectivity patterns. In this article we are investigating methodology of computation and comparisons of different causality measures, which was the subject of a number of articles devoted to it (e.g. ‐). Different measures will induce different brain structural models corresponding to the same neurological data. Comparison of different measures is necessary in order to make precise distinctions among them, which would determine: what and how exactly each of the considered measures is measuring, exact relationship between different measures in specific contexts, when, how and why they coincide or differ, and finally, when and under what conditions they are applicable and when computation of alternative measures can corroborate established connectivity patterns or discern subtle relations. Then, the ways individual computations and comparisons are done becomes important. Complexity of these models, till recently involved <24 electrodes, now with over 28 electrodes in contemporary EEG (electro encephalography) and MEG (magneto encephalography), consequently‐ operating with 216 graph links; complexity is expected to rise, towards the number of neurons >236, with 2150 synapses; increasing temporal resolution to 216 Hz, thus approaching 2180 states in 4 dimensional graphs ‐ corresponding to one hour time of brain activity. Connectivity graph measures which are properly defined, computed, compared and organized, forming firm grounds for the forthcoming complexity growth would be needed.
We briefly present fundamental concepts and causality measures which became rather dominant in brain connectivity pattern analysis. Our argumentation is common mathematical, hence, not statistically verified. We do not present statistically corroborated conclusions. We are not discussing statistical open issues, especially not how the significance thresholds are established, but we are interested in the properties these quantities have to satisfy or respect. That means: in Mathematics one counterexample is usually sufficient to show that something is going wrong, while all examples, if positive, are insufficient to prove conclusion. Analyzing computations and comparisons of different measures, we discovered that certain moments of performed comparisons and measure evaluations deserve more attention. In our considerations we separated two qualities usually treated together: connectivity from the degrees of connectedness, the former essentially much more important issue, thus obtaining much clearer situation in simplified diagrams. We use examples of published measure comparisons (‐ and other sources) to depict subtle aspects of computation and comparisons which generally need increased scrutiny and further elaboration. In order to illustrate those aspects we selected some measures which are broadly applied and somewhere compared. Especially important are the studies and achievements of Sameshima and his collaborators. We use examples of published comparisons of Directed Transfer Function ‐ DTF and Partial Directed Coherence ‐ PDC as particularly important, since numerous research teams use extensively these measures to establish brain connectivity maps. It is shown that the published results on the relationship of these two measures are partly unstable with small changes of zero thresholds (as present in a number of related published experiments), and that pictures of brain structures deduced from the cited articles need to be corrected and extended. It is shown that the neighborhood of zero is of accented importance in such evaluations and that unification of values below zero threshold is necessary as the first step in computation and comparison evaluation. Harmonization of thresholds corresponding to measures involved in comparisons is an open issue requiring mathematically reasonable solutions. Computational stability is a general demand everywhere. When we face computational instability, if we deal with models of the real world, it immediately generates semantic instability. In this context, if connectivity graphs essentially change when computational differences of arguments are within computational zero, then it is reflected semantically as proportionally unstable brain connectivity maps; this is not acceptable in any interpretation of experimental data.
Analysis of the operators involved in the measure computation and comparison proved that the spectral maximum selected as the representative invariant for both DTF and PDC before their comparison is not justified and might lead to not well founded conclusions. Suggestions for the updates of used simplification operators in computation of measures and for complementary comparisons are given. Renowned leading authors refer to “proper frequency domain counterparts to Granger causality”. Inspired by claims of Sameshima and collaborators (e.g. ), we use Geweke fundamental relation between temporal and frequency domain measures to define temporal/frequency‐domain counterparts generally in a proper way.
Investigation of PDC and DTF on synthetic (linear) models (asserting essential superiority of the former) is used in published papers as a basis in comparison of these two measures on the real neurological data. Following original methodology, on published data, we show that the deduced brain connectivity structures in this case of utmost importance are essentially unstable and could even be less different than presented and in the case of synthetic models, questioning certain steps in the computation and comparison procedures. On the other hand, following strict statistical consistency, without justified harmonization the opposite can take place and compared measure might differ to unreasonable degree. We conclude with remarks and suggestions which would improve computational representation and comparison quality and include dynamic aspects of measure comparisons. Here elaborated issues were of interest to us while investigating hardly detectable ‐ weak connectivity. This article was partly available, mostly on the conferences in recent few years.
Causality measures and their comparison
Granger’s method ‐ has been in the focus of extensive research in neuroscience, resulting in a number of method adaptations and generalizations, ever inspiring further developments and innovations specialized for specific needs. In this section we first list briefly some of the major developments which are in broad use in experimental practice, as they are commonly presented. In the sequel, we introduce Granger ‐ Geweke counterpart measure couples, correcting Bacala ‐ Sameshima  causality measure counterpart determination; then we discuss certain details related to the computation and comparisons of measures and finally we discuss the role of small sets/objects in measure estimates and comparisons. The later would provide background for the exemplary critical analysis of comparisons of key connectivity measures with corrected methodology on published data, in the following section.
More details on the method are available in the cited and other literature. A large number of scientific papers on brain area connectivity are published using causality concepts expressed as connectivity measures. Obviously, the questions of interdependence and computability of the used measures, together with some other issues are of basic importance, since directly influencing practical conclusions.
If we have three variables x(t), w(t) and y(t), if the value of x(t+1) can be determined better using past values of all the three, rather than using only x and w, then it is said that the variable y Granger causes x, or G‐causes x. Here w is a parametric variable or a set of variables.
In the bivariate case, Granger causality is expressed using linear autoregressive model as
where p is the order of linear model and Ei are the prediction errors. The model consists of the linear recursive and the stochastic component. Thus, if coefficients of y in the first equation of (1) are not all zero, we say that y G‐causes x; similarly for the variable y. The multivariate formulation was exploited more by Granger followers, Geweke [9,10] and others (e.g. [11,12]). Stated in the vector form as
Geweke  early realized importance of spectral form of G‐causality, which obtained from (2) by Fourier transform, reads as follows:
with A(0) = −I and which, solved by x gives
where H is a transfer matrix of the system.
In the bivariate case or with two blocks of variables, G‐causality measure from channel j to i at frequency λ is defined as
Geweke  elaborated conditional causality and defined a number of causality measures; we mention here his linear causality of y to x defined as
where Σ1 = var(ϵ1), Σ2 = var(E1(t)), with similar expressions for vector variables.
In frequency domain he introduced the measure of linear causality at a given frequency. Stated for two variables or two blocks of variables:
As we have noted earlier, both x and y can be vectors of variables.
Later, also in frequency domain, Kaminski and Blinowska  introduced an adaptation of Granger’s causality measure to m variables, which they called Directed Transfer Function (DTF), with formula
measuring causality from j to i at frequency λ; before, they use the same expression as in (6) for the non‐normalized DTF definition. Though extensively claimed by its authors to be superior over Granger’s measure in causality application to the brain connectivity problems, after accumulation of some experience with DTF, Kaminski et al  proposed additional connectivity measure, which should be used together with DTF, supposedly measuring direct connectivity between nodes i and j (DC) in frequency domain, defined by
where σkl(k,l = 1,…,n) are components of the covariance matrix Σ2. This measure was earlier considered by Sameshima and Baccala, e.g.  and other authors.
Baccala and Sameshima  introduced a normalized measure called Partial Directed Coherence (PDC), measuring direct influence of channel j to channel i at frequency λ, with definition
Among certain further generalizations, we mention here iPDCij(λ), the information PDC, intended to measure information flow between nodes j and i (in the sense of Information theory) by Sameshima and collaborators , which is obtained from (11) with the expansion by a factor
With the same intention for DTF, they define information DTF as following
where ρjj is the variance of the so called partialized innovation process ζ(n) defined by . Obviously, as generalizations they have respectively PDC and DTF as their special cases. They have a theorem with nine equivalent conditions characterizing absence of connectivity between two nodes j and i, of which we copy conditions 4 to 6:
This theorem is valid for two variable case. For general case, the authors announce corresponding results to be published soon. Otherwise, we note that all important conclusions in their earlier papers, especially  are affirmed again.
Aspects related to Geweke theory, computation and comparison of measures
Let us first mention improvements of Geweke causality measures, proposed in , which is based on matrix partitioning. In this way Chen, Bressler and Ding obtained corrections of Geweke conditional measures which did not suffer of deficits which the original Geweke measures had ‐ occasional negative values, smoothing peaks which were believed to be artifacts.
Above introduced measures are functions of time, or are expressed for individual frequency, when connectivity analysis for specific frequency or frequency bands is desirable. Geweke derived the fundamental agreement of the two approaches:
The relation (15) is common in Mathematics. It holds for probability measures and their densities, more generally it is common in distributions. We can use it as a definition of the counterpart measure when only one of the measures is defined. For a couple of measures F〈x,y〉 and F〈x,y〉, respectively in time and frequency domain, with parameter vector 〈x,y〉 we say that they are G‐counterparts or G‐inverse (G for Geweke) over domain ( is the set of real numbers), if they satisfy
where χD is the characteristic function of the set D.
This is slightly more general than (15), omitting D when . Thus, if one of the counterparts is given, the other can be calculated using (16). Specifically, substituting DTFij and PDCij for f in (15) and (16), we can obtain their proper time domain counterparts, the G‐inverses, which we designate by DTFij and PDCij. This would open discordance with the claim of Baccala and Sameshima () that PDC is the proper counterpart of Granger causality in frequency domain. The G‐inverse can be defined more generally than relative the aggregation operator ‐ integral form present in (16), thus realizing G‐inverse relative arbitrary aggregation operator.
In this way there are different measures related to the initial Granger‐Geweke formulations and variants. They are defined using different approaches and aspects, yet sharing the model and core of the semantics, which makes them comparable. The use of the above introduced measures in different neurological applications is broad. There are other variants and developments.
Different measures measure different properties. Some normalization and control of involved parameters are often necessary before comparison of different measures. Moreover, comparison of measures is also a measurement involving some measures and usually it can be accomplished satisfactorily in a non unique way. Some relevant issues involved in measure comparison we briefly discuss here.
When comparing measures, we can estimate their difference at a point or on a subset of a common domain, e.g. in case of well normalized measures, as important measure compatibility estimation we have
Result of measure comparison can be regarded as a kind of similarity degree (corresponding to the metrics in spaces of measures). When measuring similarity of measures, often other operators are involved e.g. normalizers, simplifiers, projections or grading, etc. We define similarity of measures somewhat more generally, in order to cover conceptually more complex approaches to measure comparison and similarity investigations. For measure μ and ν, define their mutual similarity degree by a formula fitting into the scheme
where N1 and N2 are prior normalization operators, sim is a basic similarity, P is some external operator (e.g. posterior grading), 〈i,j〉 is the graph link from i to j and D is the parameter‐set for N1 and N2. This form of comparison enables the estimation of the degree of similarity present in rather general circumstances. As usual it can be normalized. Clearly mc and mc ∗ measures are the special case of (18). Choosing the operators properly would contribute to the quality of estimation and vice versa, choosing them with insufficient care may result in reduced estimation quality and fatefulness. Computational stability, implying basic semantic stability, verified by closer inspection of Sim dependence on involved operators is a necessary requirement in this task, needing appropriate approval before inference of further conclusions. Consequently, when necessary, multiple/complementary measure comparisons should be performed.
Measures are often demanded to fulfill basic preservation properties, like monotony, cardinal monotony and translation invariance (or some approximation of it). It is important to note that measure has to satisfy substructure invariance, i.e. restriction of a measure to a substructure would not change its range; consequently, measure values on the intersection of substructures remains coherent. Further, conclusions drawn from measure computation and comparisons/similarity estimates should be invariant to some degree of fluctuations of the operators P,sim,Nk ‐ this is second requirement of semantic stability. These conditions should secure measure stability in repeated and similar experiments. Finally, comparison of measures should not violate desirable properties. Thus, small must remain small and similar has to remain similar in the procedures performing measure comparisons (literally we cannot allow that: small difference of arguments (within numerical zero) in one case is exhibited as small ‐ zero, and in the other as non‐zero. Similarly, we cannot allow that small difference of arguments in one simple computation of the predicate connected results in the values Yes (i.e. connected), while in the next computation it results in No, i.e. not connected. Structural properties of measures are usually determined on small sets/objects ‐ in the zero neighborhoods, which is why the measure on small objects (zero ideal) is of distinguished importance. This is why beside above mentioned conditions/properties we list the zero‐axioms, ZAx, either for a single considered measure or for a set of compared measures: Zero axioms (ZAx):
Z0 substructure partitioning invariance (measure on a restriction to a substructure remains coherent);
Z1 fluctuations of operators involved in measure computation and comparison need be tolerant (continuity);
Z2 in similar circumstances numeric zero (significance threshold) should be stable quantity (to allow comparability of results);
Z3 comparison of a set of measures needs prior unification (argumentation necessary) of their zero thresholds (for otherwise, what is zero for one is not zero for other measures; consequently, the measure values which are identical for one measure, are discerned as different by other measures; that must cause problems);
Z4 in similar circumstances grading should be stable quantity;
Z5 measure values which are different by ⩽ numeric‐zero should remain identical in any posterior computation/grading if applied (this is in accordance with the prior congruence on the ideal of zero measure sets);
Z6 values in any posterior computation/grading (if applied) should differ by no less than numeric‐zero and grades should be of unified diameter; in this way values in posterior grading range are harmonics of numeric‐zero;
Z7 final grading as a (small) finite projection of normalized range [0,1] needs some conceptual harmonization with the standard additivity of measures; this step should involve fuzzification;
Z8 grading should be acceptable by various aspects present in the interpretation of related experimental practice (that means that the picture obtained using a projection/grading of [0,1] range should not semantically be distant from the original picture ‐ based on the [0,1] range without grading, e.g. a sort of continuous grading).
Obviously, this list is not exhaustive and not fixed. It can be improved in different ways and the statements could be reformulated. Our intention is not to introduce further formalism in the discussed context. Certain basic mathematical principles have to be respected and we pay attention to this fact. In such way we should be able to maintain basic consistency; experimental results from repeated or similar experiments remain reasonably comparable and their comparisons reasonably stable. Measure computation and comparisons are often complex tasks consisting of a sequence of individual steps, some of which usually do not commute, demanding care and justification.
Computations and comparisons
In this section we focus our attention to the final result of the connectivity measure computations ‐ the brain connectivity directed graphs or networks, as the main model representing brain connectivity patterns. This is mainly done with concrete data, consequences are slightly more general. Due to various technological and methodological limitations, contemporary mapping of brain activity using electroencephalography and magneto encephalography operates with a few hundreds of brain signals, thus, close to mega links. No doubt, this resolution will be continuously increasing, down to a few millimeters per electrode and better, all in 3D, increasing proportionally the cardinality of connectivity graphs, as discussed earlier. In a graph we define orbits of individual nodes: the k‐th orbit of a node a will consist of nodes whose distance via a directed path from node a is k (separate for both in/out paths). We assume that the connectivity graphs exhibit direct connections of processes which are directed. This was ambition of all scientists who proposed the connectivity measures in brain analysis; this is expectation of all scientists interpreting their experiments with computation of the connectivity measures.
We rather briefly analyze some important published examples of connectivity measure computations and measure comparisons, focusing attention to the concepts and remarks from previous section. These measures are commonly used to determine brain connectivity patterns. We will exhibit erroneous or misleading conclusions in modeling of brain connectivity which is of crucial importance for experimental scientists in this area. We offer some solutions to overcome the encountered problems, aware that our argumentation could well be expanded.
Example 1. Let us first mention that a number of early articles noted the insufficiency of simple Granger ‐ two variable model applied to brain connectivity problem, with potentially wrong connectivity conclusions in case where multiple variables were related, leading to the implementation of more complex modeling, sharing the same or similar formulation to the original. The simple corrections of original Geweke measures, Geweke [9,10], by Chen et al  (both domain causality measures) were proved to be a quality resolving tool for both simulated data and neurological networks, with low zero threshold ‐ significance level (in the range 0.01 to 0.02). The corrected form is quite compatible with original on large portions of measure domain in the sense of simple measure comparison (comparison form: mc∗).
Example 2. The DTF and PDC measures are extensively used to model brain connectivity. This issue is of increasing importance. Critical comparison of results of these two measures was initiated long ago and it still needs careful investigation, since errors in conclusions could be misleading and costly.
In , there is a detailed study of DTF and statistical significance assessment, evaluating significance level related zero thresholds at 100 frequencies at 0.0045 for normalized DTF and 0.068 for the non normalized DTF. There are numerous studies related to brain connectivity problems exploiting DTF e.g. , omitting references to other connectivity measures and avoiding presentations of system structures on which DTF performs poorly (absence of transitive nodes).
Example 3. Detailed survey of PDC properties is presented by Schelter et al in , confirming its very good agreement with direct structural connectivity for simulated linear 5 argument model and a nonlinear model; here calculated statistic significance level varies from 0.01 for linear model system of order p=2, to 0.06 for order p=50, increasing to 0.12 for model order p = 200. Fitting the Var model to data generated by a Var model, some variability was introduced in PDC. With significance threshold at slightly nonconstant nearly 0.1, connectivity calculation matching well the expected connectivity pattern.
However, a number of links with local maxima (frequency) exceeding significance threshold, even by a factor of 2 were ignored as insignificant, without proper argumentation (links in Figure 3 in  designated as |Π3←1|, |Π4←1|, |Π5←1|, |Π3←2|, |Π4←2|, |Π5←2|, |Π1←2|, |Π1←3|, |Π1←4|, |Π3←4|, |Π5←4|, |Π2←5| are all with small amount of energy above significance threshold, with small bursts clearly above threshold).
Example 4. We will use the same source  as in the previous example. The application: connectivity pattern modeling tremor, involves Left EEG, Right EEG, Left EMG and Right EMG. With significance threshold at approximately 0.1, the obtained connectivity pattern is consistent with known facts, extracting the connectivity links very well from the non‐connected nodes. However, similar to the above case, the PDC diagrams corresponding to certain links (Left EMG ← Left EEG, Right EMG ← Left EMG, Left EEG ← Right EMG), all have small peaks exceeding slightly the threshold level. Again, no numeric criteria was offered to eliminate those links which should be affirmed by frequency maximum criterium which is applied in this modeling. These examples are related to the emerging conecpt of weak connectivity, the connectivity involving information transfer which is hardly discernible or even embedded in the noise energy level, examples of which were treated in certain special cases (partly published in ‐). For a nonlinear model calculated connectivity was in perfect accordance with the system definition.
Example 5. We will discuss some comparisons of normalized DTF and PDC measures (elaborated in a number of cited and other papers). As exhaustively shown by Baccala Sameshima [3,18] for the simulated models, PDC exactly determines the structural connectivity graphs of directly connected processes, while DTF is rather imprecise, mixing the direct connections with transitive influences. The results for PDC on examples with synthetic models are impressive. An example of a model emulating EEG with 6 processes is analyzed in Baccala Sameshima ; there the zero threshold is at 0.04 to 0.05. Another example with experimental data, comprising activity of structures: hippocampal CA1, somatosensory A3, motor A10 cortical areas and dorsal raphe DR, has zero threshold at 0.05, with conclusions that PDC exactly describes the direct structural connectivity, while DTF has undetermined degree of imprecision in description of direct structural connectivity.
Example 6. Analyzing structural stability, in order to emphasize importance of all steps in measure computations and comparison procedure, here we will discuss in more detail another example of PDC/DTF computation and comparison by the same authors. Analysis of an experiment focused on two shortly separated time slices: [8,10] s and [13,15] s, with frequency range [0,48] Hz, exhibiting structural connectivity changes, is given in detail by Baccala and Sameshima . Beside CA1, A3, A10, (as in the above mentioned experiment), the synchronous recordings of signals from cortical A17, hippocampal CA3 and dentate gyrus DG were included as well. Thus, this and previous experiment have a common substructure. We reproduce some of their findings/diagrams in order to be able to present our analysis.
The first time slice representation with mutual interactions of recorded structures for both PDC and DTF is given on Figure 1 (the same way of presentation is rather frequent in the literature), depicting classical coherence with solid lines; shaded spectra are respectively PDC and DTF calculations. The authors chose here common 0.20 zero‐threshold (very high: 20% of the normalized range! Four times greater than for PDC in the last mentioned experiments).These matrices were used to determine the spectral maximum for calculated PDC and DTF values for all frequencies, for each individual link, as presented in the Table 1. The matrix in the Table 1 was used subsequently to integrate and draw the connectivity diagrams for both PDC and DTF, for the first time slice, which is shown in Figure 2.
Figure 1. PDC calculated connectivity matrix on the left, shaded spectra, DTF connectivity matrix on the right, shaded spectra; DC presented with solid lines (reproduced from Baccala and Sameshima).
Table 1. The table corresponds to the first time slice of the experiment ‐ related to Figure 1; each matrix coordinate has on top the PDC spectral maximum from the spectrum at corresponding coordinates in Figure 1 left, below the spectral maximum for DTF ‐ similarly obtained from Figure 1 right; the connectivity links are sorted column vise, i.e. in the first column are A10 links towards the areas defined as row names (reproduced from Baccala and Sameshima)
The similar kind of spectral distribution matrices as in Figure 1 was published for second time slice, from which the table of maximums for both PDC and DTF was derived, which is shown in Table 2. This matrix of maximums was subsequently used to generate connectivity diagrams for the second time slice, as shown in Figure 3. The connectivity diagrams in Figures 2 and 3 depict together connectivity patterns and degree of each connectivity link by arrows in four different degrees (dashed, thin, thicker, thick and blank for zero), in the normalized [0,1] range partitioned into five values, each 0.2 in diameter.
Figure 3. Connectivity diagram, which are obtained for the second time slice, for PDC‐ left diagram, and DTF ‐ right diagram. The changes in diagrams (Figure 2 to Figure 3) depict brain dynamics in time in the described experiment. Increasing time resolution will improve our understanding of processes in the brain during experiment, thus replacing single diagrams with their time changes, i.e. time sequences of diagrams (reproduced from Baccala and Sameshima ).
Thus, with zero = 0.2, spectral maxima were extracted for each calculated signal pair, projecting ‐ grading the obtained value into the corresponding connectivity degree for each of PDC, DTF, finally considering their difference in connectivity degree to draw the conclusions on PDC/DTF performance (analysis of connectivity diagrams differences).
In terms of comparisons/similarity of measures as in (18), we can reconstruct here applied procedure (similarly in numerous other studies), which is partly implicit, as the following sequence of steps:
1. set common zero = 0.2;
2. N1 operator provides PDC spectral maximum for a given pair of inputs;
3. N2 operator provides DTF spectral maximum;
4. P operator (the same projector operator P) for both PDC and for DTF were applied as projections (the five value grading, after calculation of spectral maxima);
5. difference of the graded maxima is exhibited as visualized difference ‐ a pair of connectivity graphs depicting all pairs of signals in the respective time slices.
First, in concordance with the structural stability conditions mentioned above (on the intersection of two substructures measure is common; in repeated measurements (here, experiments) measure fluctuations must remain tolerable, i.e. obtained values coherent), we will show how rather slight variations of zero threshold, borrowed from the similar experiments cited above and presented in the cited articles influence connectivity estimates in the same example. Thus, ranging zero threshold: 0.2, 0.1 to 0.06, respectively (the values from earlier two experiments), we obtain three different connectivity difference patterns, for both time slices of this experiment.
Rather than comparing all connectivity degrees, we restrict our comparisons to a single quality: the existence of connectivity only ‐ the differences at zero level which is essential, shown in Figures 4 and 5. Complete connectivity difference diagrams are easily regenerated according to the related grading (five connectivity grades if zero = 0.2; ten grades if zero = 0.1; 33 grades when zero = 0.06).
Figure 4. The diagrams of difference in connectivity for the first time slice, shown in Figure 2, for common zero threshold equal to: 0.20, 0.10 and 0.06 (left to right) respectively, (e.g. the first diagram is complementing graphs in Figure 2 with respect to connectivity only: if we take a union of this graph links and the links in the PDC graph in Figure 2 the result is the DTF graph on the right in Figure 2; grading not shown for the simplicity). Solid lines show: DTF connected, while PDC disconnected. Thus, connectivity graph for PDC is a substructure of a corresponding graph for DTF (Note: with a data from Table 1, taking 0.06. instead of 0.20 zero threshold, for the first time slice PDC has 10 more connectivity links, while DTF obtains 8 new links).
Figure 5. The diagrams of difference in connectivity for the second time slice of the experiment as shown in Figure 3, when common zero threshold is equal to: 0.20, 0.10 and 0.06 respectively (e.g. the first diagram is complementing graphs in Figure 3 with respect to connectivity only, grading not shown for the simplicity). Solid lines: DTF connected, while PDC disconnected; dashed lines ‐ opposite. In the first two cases connectivity graph for PDC is not a substructure of a corresponding graph for DTF (note increase of connectivity links for the second time slice, similar to the first time slice).
We can notice that small changes in zero‐threshold have substantial consequences in the changes of connectivity structures and their differences. Stability analysis is mandatory whenever we have serious synthesis, i.e. when we organize and map experimental data into higher level structures with semantic significance. The brain connectivity graphs are of high importance; hence, their stability is mandatory. Second, in order to reduce or overcome some of listed problems, we shall make/suggest some changes in the measure comparison sequence, while maintaining the original procedure as much as possible:
2. apply N1 operator (provides PDC spectral maximum) for a given pair of inputs;
3. apply N2 operator (provides DTF spectral maximum) for a given pair of inputs;
4. perform zero‐ideal congruence for PDC, i.e. identify the corresponding values from previous step whose difference ⩽ zero;
5. perform zero‐ideal congruence for DTF;
6. perform zero‐ideal congruence for PDC and DTF corresponding values;
7. generate the graph of connectivity difference;
8. P operator (the same projector operator P) for both PDC and for DTF, on their respective Graphs (optional).
Clearly, in (∗∗) we have two updates of the original (∗) procedure:
•zero unification ‐ performed prior to grading, consequently, avoiding that the small (difference) becomes bigger or big, just because ranges of measures are replaced (simplified) by coarser than original smooth [0,1]‐range;
•zero‐threshold: common as in (∗), varying over values which were present in the above mentioned similar/related experiments.
Table 3. The difference of spectral maximum of PDC and spectral maximum of DTF coordinate vise for the first time slice (obtained form Table1); each matrix coordinate is the difference of the values of PDF and DTF at the same coordinate, as exhibited in Table1; the connectivity links are sorted column vise, i.e. in the first column are A10 links towards the areas defined as row names
The calculated differences in Table 3 and Table 4 are used in the corrected comparison sequence (∗∗) in the zero ‐ unification step, in order to generate more appropriate diagrams of PDC/DTF connectivity difference, which are presented in Figure 6 (for the first time slice of the experiment, for the three different zero‐threshold values) and Figure 7 (for the second time slice of the experiment).
Figure 6. The first time slice corrected connectivity comparison (i.e. using corrected procedure (**) instead of (*)). Unification of measures (based on differences in Table 3 and Table 4) prior to grading leads to the simplification of connectivity difference graphs ‐ they are substructures of graphs in Figure 4. From left to right: the difference in connectivity (corresponding to original diagrams in Figure 2) for zero threshold equal to: 0.20, 0.10 and 0.06 respectively. Solid lines: DTF connected while PDC disconnected.
Figure 7. The second time slice corrected connectivity comparison (i.e. using corrected procedure (**) instead of (*)). Unification of measures prior to grading leads to the graphs of connectivity difference, which are substructures of graphs in Figure 5. From left to right, the difference in connectivity (corresponding to original diagrams in Figure 3) for zero threshold equal to: 0.20, 0.10 and 0.06 respectively. Solid lines: DTF connected while PDC disconnected.
After the above basic convergence of the two measure comparison, we should not omit the following divergence.
Third, as mentioned above we did not essentially depart from measure computations and comparison deduced by Sameshima and collaborators in the cited papers. However, we have to notice that the Z3 is violated in the above analysis and resulting graphs, in the sense: zero thresholds (with the large difference) are unified to the max of the two without proper argumentation. Strictly: the measures have to be independently computed for each node, generating corresponding connectivity graphs. These computations have to be performed independently for each measure, using the corresponding significance level for the zero threshold, without any common zero harmonization. Finally, the agreement of the two measures is presented on the two graphs, to obtain the combined connectivity difference = measure comparison graph.
If we strictly follow the arguments related to statistic significance, with values 0.2 for PDC taken from  as above, and known value for normalized DTF (0.0045), then we would get results differing much more. In this case let the procedure corresponding to (∗) be corrected to:
1. set zero separately for each of PDC, DTF;
2. apply N1 operator (provides PDC spectral maximum) for a given pair of inputs;
3. apply N2 operator (provides DTF spectral maximum) for a given pair of inputs;
4. perform zero‐ideal congruence for PDC, i.e. identify the corresponding values from previous step whose difference ⩽ zero;
5. perform zero‐ ideal congruence for DTF;
6. generate the graph of connectivity difference.
For instance, just for the matrices in Table 1 and 2 reproduced above, we would necessarily conclude, for the connectivity only, with connectivity degree omitted, that there are numerous other links differing the resulting graphs. The strict connectivity‐only differential graphs respecting (***), for the matrices in Table 1 and 2 we present in Figure 8.
Figure 8. Difference of connectivity graphs for DTF and PDC, respectively for the data from Table1and Table2; connectivity for both DTF and PDC are determined separately for their respective independent significance thresholds. Then with the resulting connectivity graphs for each measure, the graph of connectivity difference is generated. Continuous arrows mark DTF confirmation of connectivity, while PDC confirms disconnection. Compare these two graphs to the corresponding graphs on the left side in Figures 4 and 5.
Even, when the measures are with identical value, between the two thresholds one measure will indicate connectivity, while other will deny it; example: in the first matrix
Hence, when the analyzed systems are tuned so that the compared measures measure all the links approximately identically, when we have largely departing zero thresholds for the involved measures, we can obtain arbitrarily large number of links which are zero for one and non zero for the measure with lower threshold. Moreover, when the measure values are in the opposite order, i.e. when the one with lower threshold is smaller than the other with the larger, but both being between the thresholds, the measure with smaller value will indicate connectivity, while the other with the larger value will deny it.
For example, in the second matrix (Table 2)
Obviously, harmonizing the thresholds (reducing their difference) will influence that the listed problems are diminished. This is why careful prior investigation related to Z3 is necessary.
Example 7. [D. Adams Axiom] The founders of DTF and collaborating teams have recently published serious breakthrough in the cognitive task modeling established via DTF based connectivity, e.g. ‐. A few problems are present in those findings. Connectivity of first orbits are dominating in the presented structures, e.g. Figures 9 and 10. This fact can only be related to the problematic behavior of DTF in the case when transitive connectivity nodes from higher orbits exist. However, even some small number of cascade nodes in connectivity networks are shown, with no comment on the malfunctioning of DTF when such nodes are present (example c. In Figure 9).
These authors are producing unconvincing results on connectivity. Generally, basic data from which their connectivity results are derived are hidden; in some cases when they present their data, they do it in a very reduced or unreadable form; it is immediately clear that the produced connectivity diagrams present much less connectivity arrows than there are in reality seen through the DTF.
Avoiding connectivity over full frequency domains and choosing connectivity in some of the popular frequency bands, will, clearly, lead to the reduced number of connectivity links, since a number of connectivity links related to all frequencies except those in the focus, is not mentioned at all. This is wrong.
An example of unfaithful connectivity pattern avoiding to show many more connected links, where selected‐marked connections could be determined as desirable ones, is shortly reviewed here (reproduced from , Figure 9). Without getting involved in the nature of the experimentation with cognitive modeling by connectivity patterns, we briefly touch their technical elements.
Instead of presenting connectedness, “differential” diagrams are offered, similarly to what we used in the evaluation of comparisons of measures. However, presentation of each of the investigated measure data and original frequency distributions are mandatory before differential diagrams are presented: the complete connectivity picture has to be presented prior to presentation of its filtered derivatives. As it is done in the cited and other publications, visualization of documented problematic properties of DTF might be suppressed or masked. Differential diagrams alone are insufficient, since large/very‐large can become small/negligible, thus inappropriately reflecting reality. For example, the Figures 11 and 12, which are reproduced from , show connectivity matrices with 400 frequency distributions, each ranging in the 0‐40 Hz interval, from which the conclusions on connectivity patterns related to cognitive states, i.e. “memory” versus “reasoning” tasks are “deduced”, as presented in Figure 9.
Figure 11. (reproduce from)“A typical example of the DTF functions for one subject during memory (a) condition. In each small panel, DTF (or power spectrum) is a function of frequency (from 1 to 40 Hz). Power spectra are marked on the diagonal. Off‐diagonal flows are from the electrode marked below (column) to the electrode marked (in the raw) at the left of the picture”.
Figure 12. (reproduce from)“A typical example of the DTF functions for one subject during reasoning condition (b). In each small panel, DTF (or power spectrum) is a function of frequency (from 1 to 40 Hz). Power spectra are marked on the diagonal. Off‐diagonal flows are from the electrode marked below to the electrode marked at the left of the picture”.
As one can notice immediately, there are many more connections in the (magnified) reproduced matrices than presented in the figure: according to accepted and presented criteria, the “memory tasks” exhibit over 234 connections, while “cognitive tasks” have over 217 connections in the shown 380 related spectral distributions in the matrices. The large complexity of connectedness in the obtained connectivity distributions is multiply masked; for example, the near values in different tasks are mutually evaporated; the presentation of connectedness fragmented into separately presented bands, choosing only the preferable band, reduces severely the number of really linked nodes; finally, the zero threshold positioned arbitrarily, more precisely: arbitrarily choosing what is zero, works as a selected/desired kind of a filter. Consequently, the result: a picture with a “nice” connectivity diagram as given in the Figure 9, instead of a diagram with hundreds of links if the computations were performed satisfactorily.
We are summarizing our conclusions into the following remarks:
Remark 1 (Comparison/computational sequence). The corrected comparisons of DTF and PDC, for connectivity only, as performed above, show very reduced differences of two measures for zero‐threshold from the analyzed and similar experiments (most importantly above the zero‐threshold, exhibiting which structures are connected versus those which are not), thus confirming that if analyzed with computation and comparison procedure corrections proposed here, the connectivity structures are much less different than it was demonstrated in [2,3], as presented in the graphs to the left, Figures 4 and 5, versus Figure 6 and Figure 7 (left graphs) with original common zero threshold.
Remark 2 (Aggregation prior to comparison; functionally related frequencies). The above analysis was performed, maintaining strictly the reasons and methodology performed by authors of the original analysis [2,3]. Here we have to stress that performed as it is and with our interventions in the original evaluations as well, PDC and DTF measure comparison was not performed directly on the results of these measures computations, thus, comparing directly DTF(i,j,λ) and PDC(i,j,λ) ‐ the results of measurements at each couple of nodes for each frequency in the spectrum, but, instead, as in the cited articles, the measurement of differences of these two connectivity measures was performed on their synthetic representations ‐ their prior “normalization” ‐ aggregations, obtained as the
(range(Sp) is the effective spectral ‐ frequency range) and, in the original, on their further coarser projections. In this way, in comparison of these measures the authors had substantial departure from original connectivity measurement computations for PDC and DTF. This needs further argumentation before acceptance.
Imagine a set of year final scores, with marks for 8 subjects. Then a student with all B’s would be inferior to the student with one mark A and the rest zeros. Without further argumentation and selectivity, this is usually not the accepted way of ranking. Of course, in special cases it can make sense, like in music, math, sport or military schools.
If the parts of spectrum are related to different processes which are not simply comparable in importance, for example, if one spectral band is responsible for movement detection in BCI, while the other for the deep sleep, then, depending on the application, either can be taken as representative, but most often we will not take such individual maxima of both as representing quantity for their comparisons; while, on the restrictions it can be completely reasonable. If we look closely at the corresponding coordinates in the distribution matrices in Figure 1, for the compared measures we will find examples of frequency maxima distant in the frequency domain or even in the opposite sides of spectral distributions (e.g. (5,1) ‐ first column fifth row; then (4,2), (5,2) or (6,5)).
The equation involving otherwise distant operators, even such as
becomes practically solvable when the domain becomes narrow enough. Obviously, we are approaching the question: when we have advantage of frequency measures over the temporal domain measures. In the comparison of DTF and PDC via their aggregations as explained above, much simpler insight is obtained, though, the specificity of individual frequencies is lost and comparison became essentially comparison of some sort of measure time sublimates. In such circumstances computations of connectivity measures and their comparisons need supplementary argumentation for the chosen aggregation, stability estimation and ought to be complemented with comparison of DTF and PDC time counterparts, for which we would propose their G‐inverses DTF and PDC.
We stress the demand for the comparison computation sequence correction as proposed in (∗∗) and (∗∗∗).
Remark 3 (Zero thresholds and connectedness). Maintaining original (or corrected) computational sequence in measure comparison, note that DTF computation and PDC computation are performed independently. Consequently, each of these two measures computations should apply corresponding zero threshold, as known in the literature, thus determining zero‐DTF and zero‐PDC independently for each of the computations, with independent connectedness conclusion for each measure for each pair of nodes. Then, the connectivity graphs would be statistically correct. However, if the two zeroes differ substantially, that might cause paradoxical results. Possible zero threshold unification would be highly desirable, as in e.g.  and other cited articles, but it must be well justified.
Remark 4 (Aggregations over frequency domain). Note that spectra exhibited in Figure 1 for PDC and DTF are somewhere identical, somewhere similar/proportional and somewhere hardly related at all ‐ as the consequence of different nature of these two measures (which is established by other numerous examples). The same is true for the spectral parts above zero threshold. These thresholds for the two measures cannot be unified without argumentation. Consequently, if the comparison of the two measures connectivity graphs was performed at individual frequencies or narrow frequency bands, the resulting graphs of differences in connectivity would be more fateful; they would be similar to those presented at certain frequencies, but would differ much more in the rest of the frequency domains. Obviously, connectivity at certain frequency or provably related frequencies is sufficient connectivity criterion, valid to establish that compared measures behave consistently.
Remark 5 (Brain dynamics and connectivity measures). Spectral time distributions ‐ Spectrogram like instead of spectral distributions are necessary to depict brain dynamics. In the cited articles, dynamic spectral behavior is nowhere mentioned in measure comparison considerations, but it is modestly present in some examples of brain connectivity modeling ‐ illustrating PDC applicability to the analysis focused on specific event ‐ details in [2,3]; also, in  authors recently started using matrices of spectrogram distributions instead of matrix distributions as in Figure 1. Comparison of PDC and DTF as in here analyzed articles, shows no concerns related to spectral stability /spectral dynamics and comparison results. It is clear that comparisons based on individual spectral distributions are essentially insufficient, except in proved stationary spectra, and that local time history of spectral distributions ‐ spectrograms, need to be used instead. Brain is not a static machine with a one step instruction execution.
Remark 6 (Characterization theorems for (dis)connectedness). Here we have a little sensitive play of quantifiers. By contraposition of the statement of the characterization theorem, involving information PDC and DTF as cited above, we obtain equivalence of the following conditions
and similarly with other conditions in the list.
Observe conditions (a`) and (b`). Note that λ is independently existentially quantified above. That would suggest that iPDC and iDTF simultaneously confirm the existence of connectivity from j to i. However they might do it in totally unrelated frequencies, which could make that equivalence meaningless, similarly as discussed in Remark 2. The equivalence of (a`) and (b`) apparently contradicts the nonequivalence of PDC and DTF as extensively verified in the cited very detailed analysis of Sameshima and collaborators, since these are the special cases of iPDC and iDTF. However, the statement of the theorem is true for the two variable case only, when the orbits are reduced to 1st orbits only. In this case cumulative influence reduces to the direct influence.
Authors in  do not mention zero thresholds, but in practice it has to be determined. Again, as discussed earlier and in the Remark 2, note that the same problems are equally present here. E.g. computationally we could easily have
Nobody would like that.
Remark 7 (Recent DTF based connectivity graphs with simplified orbits). In the recent publications and conference reports of research teams using DTF as connectivity measure, presenting even rather complex brain connectivity graphs involving rather numerous nodes, majority of graphs contain practically only first orbits, which is the case when deficiencies of DTF are significantly masked since cascade connectivity is hidden are not faithful, departing seriously from reality.
Remark 8 (Real‐time applicability of DTF and PDC). Both DTF and PDC measures are not applicable in real time applications like Brain Computer Interfaces ‐ BCI, where the will generated patterns in brain signals are recognized and classified by a number of direct methods. Some of methods are applicable to certain cases of weak connectivity as well.
Remark 9 (DTF connectivity versus zero threshold). The DTF based connectivity diagrams where the zero is chosen arbitrarily high or much higher than the established zero threshold (Example 7), purposely reduces the number of connectivity links by large amount, offering highly distorted facts that are established by DTF. The similar holds for synthetic spectrogram connectivity matrices. If the methodology of [3,18] was used, as in the examples 4‐6 above, one could not deduce less than 234 connected nodes in the “memory” task and not less than 217 connections in the “cognitive” task, which is strongly inconsistent with the connectivity diagrams, as reproduced here in Figures 9 and 10, derived from the reproduced matrices in Figures 11 and 12.
Remark 10 (Spectrograms instead of spectra). In the cited articles, dynamic spectral behavior is nowhere mentioned in measure comparison considerations, but it is present in some examples ‐ illustrating PDC applicability to the analysis of dynamics which is focused on specific event (details in [2,3]; also, they started using spectrogram distributions like matrices in Figure one, in ). Comparing PDC and DTF in this way there is no concern of issues related to spectral stability/spectral dynamics and comparison results. It can be demonstrated that comparisons based on individual spectra are essentially insufficient, except in proved stationary spectra, and that local time history of spectra ‐ spectrograms, need to be used instead.
Remark 11 (The DTF has been making a number of serious problems since its invention). The authors have been continuously making efforts to overcome the problems inventing further modifications of DTF, or by applying certain restrictions to their connectivity measure in order to reach connectivity diagrams which would look more faithful. Hardly had they succeeded in these intentions.
Clearly, without careful mathematical consideration and argumentation connectivity graphs, in here cited and many other published articles are of shaken fatefulness and need supplementary corroboration.
Connectivity measures are different enough that the question of their logical coherence is appropriate. This is elaborated through measure comparisons. Here the comparison of DTF and PDC measures is discussed in some detail, as an illustrative example, giving enough material for this issue to be more carefully investigated. As verified on a number of nontrivial synthetic systems, connectivity conclusions by DTF are not well founded, while PDC has good capacity in precise structural description, confirming PDC superiority to DTF measure. Quite often PDC ⩽ DTF, but it does not hold generally, hence PDC is not a general refinement of DTF and these two measures are essentially different. The problem of proper significance level determination remains partly open and needs further elaboration. When applied to real neurologic data, the methods seem to be rather sensitive on the zero ‐ threshold and comparison of PDC and DTF with proper care of small quantity maintenance gives less or more impressive PDC‐DTF difference than the results published in , enhancing the need for insights into details of measure computation and comparison procedures, with identification of non commutative operators. The zero threshold harmonization for compared measures is a difficult and challenging issue which ought’s to be solved properly.
The authors declare that they have no competing interests.
AJ has introduced the zero axioms. All authors have equally contributed to the construction of the procedures (∗), (∗∗) and (∗∗∗). All authors read and approved the final manuscript.
This work is partially supported by Serbian Ministry of Education and Science through grants III41013, ON174009 and TR36001.
Econometrica 1969, 37:424-438. Publisher Full Text
J R Stat Soc Ser A 1976, 139:246-257. Publisher Full Text
J Am Stat Assoc 1982, 77:304-313. Publisher Full Text
J Am Stat Assoc 1984, 79:907-915. Publisher Full Text
Int J Bioelectromagnetism 2008, 10(1):2-16.
Int J Bioelectromagnetism 2007, 9(3):91-102.
Comput Intell Neurosci 2009, :9.
Article ID 950403,Publisher Full Text
Brain Topogr 2011, 24:1-8..
doi:PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Babiloni F, Mattia D, Basilisco A, Astolfi L, Cincotti F, Ding L, Christine K, Sweeney J, Edgar JC, Miller GA, He B: Improved estimation of human cortical activity and connectivity with the multimodal integration of neuroelectric and hemodynamic data related to motor and cognitive tasks.
Yuan H, Doud A, Gururajan A, He B: Localization of event‐related (de)synchronization of cerebral cortex during online control of brain‐computer interface using minimum‐norm estimates in the frequency domain.
Int J Bioelectromagnetism 2007, 9(4):233-236.
Comput Intell Neurosci 2007, :7.
Article ID91651,Publisher Full Text