Why in general there is no systematic way to find counterexamples? What kind of property do they all break that lead to this? and other things

Question

We came across counterexamples in many areas of mathematics: For example

1. Sum of irrational numbers not necessary being irrational
2. The "Windmill blade" function (for lack of a better name of one of the most important counterexample in multivariable calculus that continuity $\nRightarrow$differentiability)$$z(x,y)=\left\{\begin{matrix}\frac{xy(x^2-y^2)}{x^2+y^2},(x,y)\neq(0,0)\\0,(x,y)=(0,0)\end{matrix}\right.$$
3. Linear algebra problems (e.g. Construct $2$ by $2$ matrices such that the eigenvalues of $AB$ are not the products of the eigenvalues of $A$ and $B$, and the eigenvalues of $A+B$ are not the sums of the individual eigenvalues.)

There are books wrote about counterexamples, such as this

Often, I am bad at finding counterexamples when it is the first time I came across some certain mathematical field of study, thus I tend to approach them as if I am proving something and then either find a contradiction, or encounter a step on the "attempted prove" that acts like a constraint.

For example, in answering the 3rd item, I found a subset of counterexamples that must obey the following:

$$A=\begin{pmatrix}0&b\\c&0\end{pmatrix}, B=\begin{pmatrix}0&f\\g&0\end{pmatrix}$$ $$bg+fc\neq\pm2\sqrt{bc}\sqrt{fg}\\ fg\neq \frac{g^2b}{c}\text{ or }bc\neq 0$$

Since I came across so many similar looking cases (at least in linear algebra and in some abstract algebra) it is tempting to have the following "hypothesis"

Claim: All counterexamples for a given mathematical problem obeys a set of constraint relations that is dependent to the problem.

But is that the full story?

Is there any counterexamples to the claim above?

Now for the subset of counterexamples that obey the claim above, then it should be natural the next step is to think about "optimizing" it, similar to generalising the idea of prove shortening

Is it sensible to think of trying to optimise a set of counterexample to a given problem by finding the worst possible counterexample (i.e. the one which is most nontrivial, least symmetric and breaks the most theorems (but not enough to cause it to fall outside the mathematical object in question) underlying the mathematical object that is covered by the problem?

Tldr:

What kind of fundamental property, possibly at the metamathematical level, that all counterexamples share, that makes counterexamples harder to find in a systematic way than constructing proofs?

To elaborate Qiaochu Yuan said in item 3:

Finding counterexamples is something of a dark art; I have seen literally no mathematical writing of any kind which explicitly discusses how one might go about doing it, even though it is quite an important mathematical skill. Here are some thoughts off the top of my head.

So it seems harder than proofs because there is no known systematic way to do it, other than intuition and trial and error

Understand why this is the case will help me to understand why it is difficult to devise a systematic way to find nontrivial counterexamples such as the windmill blade function

Answer 1

I would argue neither is harder in general, or at least there's no objective way of making such a judgement.

Finding a counterexample doesn't necessarily require an ingenious guess though, you can find one in a relatively systematic fashion. When proving a theorem, the main question you are trying to answer is "why do these hypotheses imply the conclusion?" You may try to come up with a series of deductions that eventually leads to the result.

Constructing a counterexample is just the opposite of this. You are asking the question "what assumption in my hypothesis is too weak to imply the conclusion and what would be instead sufficient?" You then construct an example which only just satisfies the weak hypothesis and see if it works.

Granted this is only one way of constructing counterexamples (another consisting of guessing until you find one). For example in analysis, if continuity alone is insufficient to prove a claim, you may try a function that is continuous but not differentiable or one that is unbounded, for example. Here you may have a pre-existing toolkit of counterexamples which satisfy property $X$, but not $Y$, which can be tested against the claim.

Of course, this does not make this process easy, but that's the same with proving something. In general, there are no systematic ways of solving problems and at some point, you will need a logical jump or a piece of insight. Common tricks exist though, including so-called standard counterexamples.

Answer 2

I completely disagree with your statement that it is easier to prove things than it is to find counterexamples. There are statements that are easy to prove

For every real valued function $f$, there exists a function $g$ such that $\forall x\in \mathbb R: f(x)<(g)$

and statements hard to prove.

There are no integer solutions to the equation $$a^n+b^n=c^n$$ for $n>2$

And there are statements with trivial counterexamples

For all real numbers $x$, there exists such a number $y$ that $y^2=x$

and statements with nontrivial counterexamples, like your windmill function.

Answer 3

I think the opposite...disproving is easier than proving...reason is that for proving something you need to show that all cases under the given condition is going to be true...but for disproving something you need to provide just one example which is not in accordance with the given mathematical statement. Also before proving anything you will have to deal with many examples,find out the common among them and proceed.Disproving is much easier....you don't need to do all these.

Answer 4

You are asking an incredibly broad question here. My best answer is to question your question. What exactly do you mean by a counter-example? We usually mean something that goes against our intuition. Formally though, a counterexample is just something that goes against a claim e.g. "All integers are greater than $0$" has the obvious counter example of $-1$. OK, so that's obvious, but it's a good (counter-)example of how not all "formal" counter-examples are really useful or insightful. So then we must ask, what is a useful or insightful counter-example? And from here, the question becomes one of opinion, where we must decide what is intuitive, useful, insightful etc. And so I'll stop here lest I get bogged down in my own tedious opinions.