Existence of positive solutions for a critical nonlinear Schrödinger equation with vanishing or coercive potentials

Abstract

In this paper we investigate the existence of positive solutions for the following nonlinear Schrödinger equation:

$$-\mathrm{△}u+V(x)u=K(x){|u|}^{p-2}u\text{in }{\mathbb{R}}^N,$$

where $V(x)\sim a{|x|}^{-b}$ and $K(x)\sim \mu {|x|}^{-s}$ as $|x|\to \mathrm{\infty }$ with $0<a,\mu <+\mathrm{\infty }$, $b<2$, $b\ne 0$, $0<\frac{s}{b}<1$ and $p=2(N-2s/b)/(N-2)$.

MSC:35J20, 35J60.

1 Introduction and statement of results

In this paper, we consider the following semilinear elliptic equation:

$$-\mathrm{△}u+V(x)u=K(x){|u|}^{p-2}u\text{in }{\mathbb{R}}^N,$$

(1.1)

where $N\ge 3$. The exponent

$$p=2(N-\frac{2s}{b})/(N-2)$$

(1.2)

with the real numbers b and s satisfying

$$b<2,b\ne 0,0<\frac{s}{b}<1.$$

(1.3)

By this definition, $2<p<2^{\ast }:=2N/(N-2)$.

With respect to the functions V and K, we assume that

(A₁) $V,K\in C({\mathbb{R}}^N)$ for every $x\in {\mathbb{R}}^N$, $V(x)>0$ and $K(x)>0$.

(A₂) There exist $0<a<\mathrm{\infty }$ and $0<\mu <\mathrm{\infty }$ such that

$$\underset{|x|\to \mathrm{\infty }}{lim}{|x|}^{b}V(x)=a\text{and}\underset{|x|\to \mathrm{\infty }}{lim}{|x|}^{s}K(x)=\mu .$$

(1.4)

A typical example for Eq. (1.1) with V and K satisfying (A₁) and (A₂) is the equation

$$-\mathrm{△}u+\frac{a}{{(1+|x|)}^b}u=\frac{\mu }{{(1+|x|)}^s}{|u|}^{p-2}u\text{in }{\mathbb{R}}^N.$$

(1.5)

When $0<b<2$, the potentials are vanishing at infinity and when $b<0$, the potentials are coercive.

Equation (1.1) arises in various applications, such as chemotaxis, population genetics, chemical reactor theory and the study of standing wave solutions of certain nonlinear Schrödinger equations. Therefore, they have received growing attention in recent years (one can see, e.g., [1–6] and [7–10] for reference).

Under the above assumptions, Eq. (1.1) has a natural variational structure. For an open subset Ω in ${\mathbb{R}}^N$, let $C_0^{\mathrm{\infty }}(\mathrm{\Omega })$ be the collection of smooth functions with a compact support set in Ω. Let E be the completion of $C_0^{\mathrm{\infty }}({\mathbb{R}}^N)$ with respect to the inner product

$${(u,v)}_E={\int }_{{\mathbb{R}}^N}\mathrm{\nabla }u\mathrm{\nabla }vdx+{\int }_{{\mathbb{R}}^N}V(x)uvdx.$$

From assumptions (A₁) and (A₂), we deduce that

$${\left({\int }_{{\mathbb{R}}^N}\frac{{|u|}^2}{{(1+|x|)}^b}dx\right)}^{1/2}\text{and}{({\int }_{{\mathbb{R}}^N}V(x){|u|}^{2}dx)}^{1/2}$$

are two equivalent norms in the space

$$L_V^2\left({\mathbb{R}}^N\right)=\left\{u\text{ is measurable in }{\mathbb{R}}^N\right|{\int }_{{\mathbb{R}}^N}V(x){|u|}^{2}dx<+\mathrm{\infty }\}.$$

Therefore, there exists $B_1>0$ such that

$${\left({\int }_{{\mathbb{R}}^N}\frac{{|u|}^2}{{(1+|x|)}^b}dx\right)}^{1/2}\le B_1{({\int }_{{\mathbb{R}}^N}V(x){|u|}^{2}dx)}^{1/2}.$$

Moreover, assumptions (A₁) and (A₂) imply that there exists $B_2>0$ such that

$$K(x)\le B_2{(1+|x|)}^{-s},\mathrm{\forall }x\in {\mathbb{R}}^N.$$

Then, by the Hölder and Sobolev inequalities (see, e.g., [[11], Theorem 1.8]), we have, for every $u\in C_0^{\mathrm{\infty }}({\mathbb{R}}^N)$,

$$\begin{array}{rcl}{({\int }_{{\mathbb{R}}^N}K(x){|u|}^{p}dx)}^{\frac{1}{p}}& \le & C{\left({\int }_{{\mathbb{R}}^N}\frac{{|u|}^p}{{(1+|x|)}^s}dx\right)}^{\frac{1}{p}}\\ =& C{({\int }_{{\mathbb{R}}^N}\frac{{|u|}^{\frac{2s}{b}}}{{(1+|x|)}^s}\cdot {|u|}^{p-\frac{2s}{b}}dx)}^{\frac{1}{p}}\\ \le & C{\left({\int }_{{\mathbb{R}}^N}\frac{{|u|}^2}{{(1+|x|)}^b}dx\right)}^{\frac{s}{pb}}{\left({\int }_{{\mathbb{R}}^N}{|u|}^{2^{\ast }}dx\right)}^{\frac{1}{p}(1-\frac{s}{b})}\\ \le & C{\left({\int }_{{\mathbb{R}}^N}\frac{{|u|}^2}{{(1+|x|)}^b}dx\right)}^{\frac{s}{pb}}{\left({\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx\right)}^{\frac{2^{\ast }}{2p}(1-\frac{s}{b})}\\ =& C{\left({\int }_{{\mathbb{R}}^N}\frac{{|u|}^2}{{(1+|x|)}^b}dx\right)}^{\frac{1}{2}\cdot \frac{2s}{pb}}{\left({\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx\right)}^{\frac{1}{2}\cdot (1-\frac{2s}{pb})}\\ \le & C{({\int }_{{\mathbb{R}}^N}V(x){|u|}^{2}dx)}^{\frac{1}{2}\cdot \frac{2s}{pb}}{\left({\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx\right)}^{\frac{1}{2}\cdot (1-\frac{2s}{pb})},\end{array}$$

where $C>0$ is a constant independent of u. It follows that there exists a constant $C^{\prime }>0$ such that

$${({\int }_{{\mathbb{R}}^N}K(x){|u|}^{p}dx)}^{1/p}\le C^{\prime }{\left({\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx\right)}^{1/2}+C^{\prime }{({\int }_{{\mathbb{R}}^N}V(x){|u|}^{2}dx)}^{1/2}.$$

This implies that E can be embedded continuously into the weighted $L^p$-space

$$L_K^p\left({\mathbb{R}}^N\right)=\left\{u\text{ is measurable in }{\mathbb{R}}^N\right|{\int }_{{\mathbb{R}}^N}K(x){|u|}^{p}dx<+\mathrm{\infty }\}.$$

Then the functional

$$\mathrm{\Phi }(u)=\frac{1}{2}{\parallel u\parallel }_E^2-\frac{1}{p}{\int }_{{\mathbb{R}}^N}K(x){|u|}^{p}dx,u\in E,$$

is well defined in E. And it is easy to check that Φ is a $C^2$ functional and the critical points of Φ are solutions of (1.1) in E.

In a recent paper [12], Alves and Souto proved that the space E can be embedded compactly into $L_K^p({\mathbb{R}}^N)$ if $0<b<2$ and $2(N-2s/b)/(N-2)<p<2^{\ast }$ and Φ satisfies the Palais-Smale condition consequently. Then, by using the mountain pass theorem, they obtained a nontrivial solution for Eq. (1.1). Unfortunately, when $p=2(N-2s/b)/(N-2)$, the embedding of E into $L_K^p({\mathbb{R}}^N)$ is not compact and Φ no longer satisfies the Palais-Smale condition. Therefore, the ‘standard’ variational methods fail in this case. From this point of view, $p=2(N-2s/b)/(N-2)$ should be seen as a kind of critical exponent for Eq. (1.1). If the potentials V and K are restricted to the class of radially symmetric functions, ‘compactness’ of such a kind is regained and ‘standard’ variational approaches work (see [5] and [6]). However, this method does not seem to apply to the more general equation (1.1) where K and V are non-radially symmetric functions.

It is not easy to deal with Eq. (1.1) directly because there are no known approaches that can be used directly to overcome the difficulty brought by the loss of compactness. However, in this paper, through an interesting transformation, we find an equivalent equation for Eq. (1.1) (see Eq. (2.9) in Section 2). This equation has the advantages that its Palais-Smale sequence can be characterized precisely through the concentration-compactness principle (see Theorem 5.1), and it possesses partial compactness (see Corollary 5.8). By means of these advantages, a positive solution for this equivalent equation and then a corresponding positive solution for Eq. (1.1) are obtained.

Before stating our main result, we need to give some definitions.

Let

$$V_{\ast }(x)={|x|}^{\frac{2b}{2-b}}V\left({|x|}^{\frac{b}{2-b}}x\right)+C_b{|x|}^{-2},$$

(1.6)

where

$$C_b=\frac{b}{4}(1-\frac{b}{4}){(N-2)}^2$$

(1.7)

and

$$K_{\ast }(x)={|x|}^{\frac{2s}{2-b}}K\left({|x|}^{\frac{b}{2-b}}x\right).$$

(1.8)

Let $H^1({\mathbb{R}}^N)$ be the Sobolev space endowed with the norm and the inner product

$$\parallel u\parallel ={({\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+{\int }_{{\mathbb{R}}^N}{u}^{2}dx)}^{1/2}\text{and}(u,v)={\int }_{{\mathbb{R}}^N}(\mathrm{\nabla }u\cdot \mathrm{\nabla }v+uv)dx,$$

respectively, and let $L^p({\mathbb{R}}^N)$ be the function space consisting of the functions on ${\mathbb{R}}^N$ that are p-integrable. Since $2<p<2^{\ast }$, $H^1({\mathbb{R}}^N)$ can be embedded continuously into $L^p({\mathbb{R}}^N)$. Therefore, the infimum

$$\underset{v\in H^1({\mathbb{R}}^N)\setminus \{0\}}{inf}\frac{{\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }v|}^{2}dx+a{\int }_{{\mathbb{R}}^N}{v}^{2}dx}{{({\int }_{{\mathbb{R}}^N}{|v|}^{p}dx)}^{2/p}}>0.$$

(1.9)

We denote this infimum by $S_p$.

Our main result reads as follows.

Theorem 1.1 Under assumptions (A₁) and (A₂), if b, s and p satisfy (1.3) and (1.2) and

$$\begin{array}{r}\underset{u\in H^1({\mathbb{R}}^N)\setminus \{0\}}{inf}\frac{{\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }u|}^2}{{|x|}^2}dx+{\int }_{{\mathbb{R}}^N}{V}_{\ast }(x){|u|}^{2}dx}{{({\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){|u|}^{p}dx)}^{2/p}}\\ <{(1-b/2)}^{\frac{p-2}{p}}{\mu }^{-\frac{2}{p}}{S}_p,\end{array}$$

(1.10)

then Eq. (1.1) has a positive solution $u\in E$.

Remark 1.2 We should emphasize that condition (1.10) can be satisfied in many situations. For $r>0$, let $R_r=\{x\in {\mathbb{R}}^N\mid r/2<|x|<r\}$ and $H_0^1(R_r)$ be the closure of $C_0^{\mathrm{\infty }}(R_r)$ in $H^1({\mathbb{R}}^N)$. Under assumptions (A₁) and (A₂), we have

$$\underset{u\in H_0^1(R_r)\setminus \{0\}}{inf}\frac{{\int }_{R_r}{|\mathrm{\nabla }u|}^{2}dx}{{({\int }_{R_r}{K}_{\ast }(x){|u|}^{p}dx)}^{2/p}}\to 0,\text{as }r\to +\mathrm{\infty }.$$

Then, for any $ϵ>0$, there exist $r_{ϵ}>0$ and $u_{ϵ}\in H_0^1(R_r)\setminus \{0\}$ such that

$$\frac{{\int }_{R_r}{|\mathrm{\nabla }{u}_{ϵ}|}^{2}dx}{{({\int }_{R_r}{K}_{\ast }(x){|u_{ϵ}|}^{p}dx)}^{2/p}}<ϵ.$$

It follows from this inequality and ${\int }_{R_r}\frac{{|x\cdot \mathrm{\nabla }{u}_{ϵ}|}^2}{{|x|}^2}dx\le {\int }_{R_r}{|\mathrm{\nabla }{u}_{ϵ}|}^{2}dx$ that if ${sup}_{R_r}{V}_{\ast }$ is small enough such that

$$\frac{{\int }_{R_r}{V}_{\ast }(x){|u_{ϵ}|}^{2}dx}{{({\int }_{R_r}{K}_{\ast }(x){|u_{ϵ}|}^{p}dx)}^{2/p}}<ϵ,$$

then

$$\begin{array}{r}\frac{{\int }_{R_r}{|\mathrm{\nabla }{u}_{ϵ}|}^{2}dx+(\frac{b^2}{4}-b){\int }_{R_r}\frac{{|x\cdot \mathrm{\nabla }{u}_{ϵ}|}^2}{{|x|}^2}dx+{\int }_{R_r}{V}_{\ast }(x){|u_{ϵ}|}^{2}dx}{{({\int }_{R_r}{K}_{\ast }(x){|u_{ϵ}|}^{p}dx)}^{2/p}}\\ <(2+|\frac{b^2}{4}-b|)ϵ.\end{array}$$

This implies that (1.10) is satisfied if ϵ is chosen such that $(2+|\frac{b^2}{4}-b|)ϵ<{(1-b/2)}^{\frac{p-2}{p}}{\mu }^{-\frac{2}{p}}{S}_p$.

Notations Let X be a Banach space and $\phi \in C^1(X,\mathbb{R})$. We denote the Fréchet derivative of φ at u by ${\phi }^{\prime }(u)$. The Gateaux derivative of φ is denoted by $〈{\phi }^{\prime }(u),v〉$, $\mathrm{\forall }u,v\in X$. By → we denote the strong and by ⇀ the weak convergence. For a function u, $u^{+}$ denotes the functions $max\{u(x),0\}$. The symbol ${\delta }_{ij}$ denotes the Kronecker symbol:

$${\delta }_{ij}=\{\begin{array}{cc}1,\hfill & i=j,\hfill \\ 0,\hfill & i\ne j.\hfill \end{array}$$

We use $o(h)$ to mean $o(h)/|h|\to 0$ as $|h|\to 0$.

2 An equivalent equation for Eq. (1.1)

For $x\in {\mathbb{R}}^N$, let $y={|x|}^{-b/2}x$. To u, a $C^2$ function in ${\mathbb{R}}^N$, we associate a function v, a $C^2$ function in ${\mathbb{R}}^N\setminus \{0\}$, by the transformation

$$u(x)={|x|}^{-\frac{b}{4}(N-2)}v({|x|}^{-\frac{b}{2}}{x}_1,\dots ,{|x|}^{-\frac{b}{2}}{x}_N).$$

(2.1)

Lemma 2.1 Under the above assumptions,

$${\mathrm{△}}_{x}u(x)={|y|}^{-\frac{b(N+2)}{2(2-b)}}(\sum _{i,j=1}^N\frac{\partial }{\partial y_j}(A_{ij}(y)\frac{\partial v}{\partial y_i})-\frac{C_b}{{|y|}^2}v),$$

(2.2)

where

$$A_{ij}(y)={\delta }_{ij}+(\frac{b^2}{4}-b)\frac{y_i{y}_j}{{|y|}^2},i,j=1,\dots ,N.$$

(2.3)

Proof Let $r=|x|$. By direct computations,

$$\frac{\partial u}{\partial x_i}=r^{-\frac{b(N-2)}{4}-\frac{b}{2}}\frac{\partial v}{\partial y_i}-\frac{b}{2}{r}^{-\frac{b(N-2)}{4}-\frac{b}{2}-2}{x}_i\sum _{j=1}^N{x}_j\frac{\partial v}{\partial y_j}-\frac{b}{4}(N-2)r^{-\frac{b(N-2)}{4}-2}{x}_{i}v$$

(2.4)

and

$$\begin{array}{rcl}\frac{{\partial }^{2}u}{\partial x_i^2}& =& -\frac{bN}{2}{r}^{-\frac{b(N-2)}{4}-\frac{b}{2}-2}{x}_i\frac{\partial v}{\partial y_i}+r^{-\frac{b(N-2)}{4}-b}\frac{{\partial }^{2}v}{\partial y_i^2}-b{r}^{-\frac{b(N-2)}{4}-b-2}\sum _{j=1}^N{x}_j{x}_i\frac{{\partial }^{2}v}{\partial y_j\partial y_i}\\ +(\frac{b^2}{4}(N-1)+b)r^{-\frac{b(N-2)}{4}-\frac{b}{2}-4}{x}_i^2\sum _{j=1}^N{x}_j\frac{\partial v}{\partial y_j}\\ -\frac{b}{2}{r}^{-\frac{b(N-2)}{4}-\frac{b}{2}-2}\sum _{j=1}^N{x}_j\frac{\partial v}{\partial y_j}+\frac{b^2}{4}{r}^{-\frac{b(N-2)}{4}-b-4}{x}_i^2\sum _{j,k=1}^N{x}_j{x}_k\frac{{\partial }^{2}v}{\partial y_j\partial y_k}\\ +\frac{b}{4}(N-2)(\frac{b}{4}(N-2)+2)r^{-\frac{b}{4}(N-2)-4}{x}_i^{2}v-\frac{b}{4}(N-2)r^{-\frac{b}{4}(N-2)-2}v.\end{array}$$

Then

$$\begin{array}{rcl}{\mathrm{△}}_{x}u& =& \sum _{i=1}^N\frac{{\partial }^{2}u}{\partial x_i^2}\\ =& r^{-\frac{b(N-2)}{4}-b}\{{\mathrm{△}}_{y}v+(\frac{b^2}{4}-b)r^{-2}\sum _{i,j=1}^N{x}_i{x}_j\frac{{\partial }^{2}v}{\partial y_i\partial y_j}\\ +(\frac{b^2}{4}-b)(N-1)r^{\frac{b}{2}-2}\sum _{i=1}^N{x}_i\frac{\partial v}{\partial y_i}-\frac{b}{4}(1-\frac{b}{4}){(N-2)}^2{r}^{b-2}v\}.\end{array}$$

(2.5)

Since $y={|x|}^{-b/2}x$, we have $r={|y|}^{\frac{2}{2-b}}$ and $x_i={|y|}^{\frac{b}{2-b}}{y}_i$, $1\le i\le N$. Then

$$\begin{array}{r}{r}^{-2}\sum _{i,j=1}^N{x}_i{x}_j\frac{{\partial }^{2}v}{\partial y_i\partial y_j}+(N-1)r^{\frac{b}{2}-2}\sum _{i=1}^N{x}_i\frac{\partial v}{\partial y_i}\\ ={|y|}^{-2}\sum _{i,j=1}^N{y}_i{y}_j\frac{{\partial }^{2}v}{\partial y_i\partial y_j}+(N-1){|y|}^{-2}\sum _{i=1}^N{y}_i\frac{\partial v}{\partial y_i}\\ =\sum _{i,j=1}^N\frac{\partial }{\partial y_j}\left(\frac{y_i{y}_j}{{|y|}^2}\frac{\partial v}{\partial y_i}\right).\end{array}$$

(2.6)

Substituting (2.6) and $r={|y|}^{\frac{2}{2-b}}$ into (2.5) results in

$$\begin{array}{rcl}{\mathrm{△}}_{x}u(x)& =& {|y|}^{-\frac{b(N+2)}{2(2-b)}}({\mathrm{△}}_{y}v+(\frac{b^2}{4}-b)\sum _{i,j=1}^N\frac{\partial }{\partial y_j}\left(\frac{y_i{y}_j}{{|y|}^2}\frac{\partial v}{\partial y_i}\right)-\frac{C_b}{{|y|}^2}v)\\ =& {|y|}^{-\frac{b(N+2)}{2(2-b)}}(\sum _{i,j=1}^N\frac{\partial }{\partial y_j}(A_{ij}(y)\frac{\partial v}{\partial y_i})-\frac{C_b}{{|y|}^2}v).\end{array}$$

 □

Let

$$\begin{array}{r}{H}_{\mathrm{loc}}^1\left({\mathbb{R}}^N\right)\\ =\{u|\text{ for every bounded domain }\mathrm{\Omega }\subset {\mathbb{R}}^N,{\int }_{\mathrm{\Omega }}{|\mathrm{\nabla }u|}^{2}dx+{\int }_{\mathrm{\Omega }}{u}^{2}dx<+\mathrm{\infty }\}.\end{array}$$

(2.7)

From the classical Hardy inequality (see, e.g., [[13], Lemma 2.1]), we deduce that for every bounded $C^1$ domain $\mathrm{\Omega }\subset {\mathbb{R}}^N$, there exists $C_{\mathrm{\Omega }}>0$ such that, for every $u\in H_{\mathrm{loc}}^1({\mathbb{R}}^N)$,

$${\int }_{\mathrm{\Omega }}\frac{u^2}{{|x|}^2}dx\le C_{\mathrm{\Omega }}({\int }_{\mathrm{\Omega }}{|\mathrm{\nabla }u|}^{2}dx+{\int }_{\mathrm{\Omega }}{u}^{2}dx).$$

(2.8)

Theorem 2.2 If $v\in H_{\mathrm{loc}}^1({\mathbb{R}}^N)$ is a weak solution of the equation

$$-\sum _{i,j=1}^N\frac{\partial }{\partial y_j}(A_{ij}(y)\frac{\partial v}{\partial y_i})+V_{\ast }v=K_{\ast }{|v|}^{p-2}v\mathit{\text{in }}{\mathbb{R}}^N,$$

(2.9)

i.e., for every $\psi \in C_0^{\mathrm{\infty }}({\mathbb{R}}^N)$,

$${\int }_{{\mathbb{R}}^N}\sum _{i,j=1}^N{A}_{ij}(y)\frac{\partial v}{\partial y_i}\frac{\partial \psi }{\partial y_j}dy+{\int }_{{\mathbb{R}}^N}{V}_{\ast }(y)v\psi dy={\int }_{{\mathbb{R}}^N}{K}_{\ast }(y){|v|}^{p-2}v\psi dy,$$

(2.10)

and u is defined by (2.1), then $u\in H_{\mathrm{loc}}^1({\mathbb{R}}^N)$ and it is a weak solution of (1.1), i.e., for every $\phi \in C_0^{\mathrm{\infty }}({\mathbb{R}}^N)$,

$${\int }_{{\mathbb{R}}^N}\mathrm{\nabla }u\mathrm{\nabla }\phi dx+{\int }_{{\mathbb{R}}^N}V(x)u\phi dx={\int }_{{\mathbb{R}}^N}K(x){|u|}^{p-2}u\phi dx.$$

(2.11)

Proof Using the spherical coordinates

$$\begin{array}{r}{x}_1=rcos{\sigma }_1,\\ x_2=rsin{\sigma }_{1}cos{\sigma }_2,\\ \cdots \\ x_j=rsin{\sigma }_{1}sin{\sigma }_2\cdots sin{\sigma }_{j-1}cos{\sigma }_j,2\le j\le N-1,\\ \cdots \\ x_N=rsin{\sigma }_{1}sin{\sigma }_2\cdots sin{\sigma }_{N-2}sin{\sigma }_{N-1},\end{array}$$

where $0\le {\sigma }_j<\pi$, $j=1,2,\dots ,N-2$, $0\le {\sigma }_{N-1}<2\pi$, we have

$$dx=r^{N-1}f(\sigma )drd{\sigma }_1\cdots d{\sigma }_{N-1},$$

where $f(\sigma )={sin}^{N-2}{\sigma }_1{sin}^{N-3}{\sigma }_2\cdots sin{\sigma }_{N-2}$. Recall that $y={|x|}^{-\frac{b}{2}}x$. Let $R=|y|$. Then $r=R^{\frac{2}{2-b}}$ and

$$\begin{array}{rcl}dx& =& r^{N-1}f(\sigma )drd{\sigma }_1\cdots d{\sigma }_{N-1}=R^{\frac{2(N-1)}{2-b}}f(\sigma )d\left(R^{\frac{2}{2-b}}\right)d{\sigma }_1\cdots d{\sigma }_{N-1}\\ =& \frac{2}{2-b}{R}^{\frac{2N}{2-b}-1}f(\sigma )dRd{\sigma }_1\cdots d{\sigma }_{N-1}=\frac{2}{2-b}{|y|}^{\frac{bN}{2-b}}dy.\end{array}$$

(2.12)

Here, we used $dy=R^{N-1}f(\sigma )dRd{\sigma }_1\cdots d{\sigma }_{N-1}$ in the last inequality above. From (2.4), (2.12) and (2.8), we deduce that there exists $C>0$ such that for every bounded domain $\mathrm{\Omega }\subset {\mathbb{R}}^N$,

$$\begin{array}{rcl}{\int }_{\mathrm{\Omega }}{\left|\frac{\partial u}{\partial x_i}\right|}^{2}dx& \le & C{\int }_{\mathrm{\Omega }}{r}^{-\frac{b(N-2)}{2}-b}{\left(\frac{\partial v}{\partial y_i}\left({|x|}^{-b/2}x\right)\right)}^{2}dx\\ +C{\int }_{\mathrm{\Omega }}{r}^{-\frac{b(N-2)}{2}-b-4}{\left(x_i\sum _{j=1}^N{x}_j\frac{\partial v}{\partial y_j}\left({|x|}^{-b/2}x\right)\right)}^{2}dx\\ +C{\int }_{\mathrm{\Omega }}{r}^{-\frac{b(N-2)}{2}-4}{x}_i^2{v}^2\left({|x|}^{-b/2}x\right)dx\\ =& \frac{2C}{2-b}{\int }_{\mathrm{\Omega }}{\left(\frac{\partial v(y)}{\partial y_i}\right)}^{2}dy+\frac{2C}{2-b}{\int }_{\mathrm{\Omega }}{\left(\frac{y_i}{|y|}\sum _{j=1}^N\frac{y_j}{|y|}\frac{\partial v(y)}{\partial y_j}\right)}^{2}dy\\ +\frac{2C}{2-b}{\int }_{\mathrm{\Omega }}{|y|}^{-4}{y}_i^2{v}^2(y)dy\\ \le & C^{″}({\int }_{\mathrm{\Omega }}{|\mathrm{\nabla }v|}^{2}dy+{\int }_{\mathrm{\Omega }}\frac{v^2}{{|y|}^2}dy)<+\mathrm{\infty }.\end{array}$$

Moreover,

$${\int }_{\mathrm{\Omega }}{u}^{2}dx={\int }_{\mathrm{\Omega }}{|x|}^{-\frac{b}{2}(N-2)}{v}^2\left({|x|}^{-\frac{b}{2}}x\right)dx=\frac{2}{2-b}{\int }_{\mathrm{\Omega }}{|y|}^{\frac{2b}{2-b}}{v}^2(y)dy<+\mathrm{\infty }.$$

Therefore, $u\in H_{\mathrm{loc}}^1({\mathbb{R}}^N)$. Then, to prove that u satisfies (2.11) for every $\phi \in C_0^{\mathrm{\infty }}({\mathbb{R}}^N)$, it suffices to prove that (2.11) holds for every $\phi \in C_0^{\mathrm{\infty }}({\mathbb{R}}^N\setminus \{0\})$. For $\phi \in C_0^{\mathrm{\infty }}({\mathbb{R}}^N\setminus \{0\})$, let $\psi \in C_0^{\mathrm{\infty }}({\mathbb{R}}^N\setminus \{0\})$ be such that

$$\phi (x)={|x|}^{-\frac{b}{4}(N-2)}\psi \left({|x|}^{-\frac{b}{2}}x\right).$$

By using the divergence theorem and Lemma 2.1, we get that

$$\begin{array}{r}{\int }_{{\mathbb{R}}^N}\mathrm{\nabla }u\mathrm{\nabla }\phi dx\\ =-{\int }_{{\mathbb{R}}^N}u\mathrm{△}\phi dx\\ =-{\int }_{{\mathbb{R}}^N}u\cdot {|y|}^{-\frac{b(N+2)}{2(2-b)}}(\sum _{i,j=1}^N\frac{\partial }{\partial y_j}(A_{ij}(y)\frac{\partial \psi }{\partial y_i})-\frac{C_b}{{|y|}^2}\psi )dx\\ =-{\int }_{{\mathbb{R}}^N}{|x|}^{-\frac{b}{4}(N-2)}v\left({|x|}^{-\frac{b}{2}}x\right)\cdot {|y|}^{-\frac{b(N+2)}{2(2-b)}}(\sum _{i,j=1}^N\frac{\partial }{\partial y_j}(A_{ij}(y)\frac{\partial \psi }{\partial y_i})-\frac{C_b}{{|y|}^2}\psi )dx\\ =-{\int }_{{\mathbb{R}}^N}{|y|}^{-\frac{b(N-2)}{2(2-b)}}v(y)\cdot {|y|}^{-\frac{b(N+2)}{2(2-b)}}(\sum _{i,j=1}^N\frac{\partial }{\partial y_j}(A_{ij}(y)\frac{\partial \psi }{\partial y_i})-\frac{C_b}{{|y|}^2}\psi )\frac{2}{2-b}{|y|}^{\frac{bN}{2-b}}dy\\ =-\frac{2}{2-b}{\int }_{{\mathbb{R}}^N}v\cdot (\sum _{i,j=1}^N\frac{\partial }{\partial y_j}(A_{ij}(y)\frac{\partial \psi }{\partial y_i})-\frac{C_b}{{|y|}^2}\psi )dy\\ =\frac{2}{2-b}{\int }_{{\mathbb{R}}^N}\sum _{i,j=1}^N{A}_{ij}(y)\frac{\partial v}{\partial y_i}\frac{\partial \psi }{\partial y_j}dy-\frac{2C_b}{2-b}{\int }_{{\mathbb{R}}^N}\frac{v\psi }{{|y|}^2}dy.\end{array}$$

Moreover,

$$\begin{array}{r}{\int }_{{\mathbb{R}}^N}V(x)u\phi dx\\ =\frac{2}{2-b}{\int }_{{\mathbb{R}}^N}V\left({|y|}^{\frac{b}{2-b}}y\right)u\left({|y|}^{\frac{b}{2-b}}y\right)\phi \left({|y|}^{\frac{b}{2-b}}y\right){|y|}^{\frac{bN}{2-b}}dy\\ =\frac{2}{2-b}{\int }_{{\mathbb{R}}^N}{|y|}^{\frac{2b}{2-b}}V\left({|y|}^{\frac{b}{2-b}}y\right)\cdot {|y|}^{\frac{b(N-2)}{2(2-b)}}u\left({|y|}^{\frac{b}{2-b}}y\right)\cdot {|y|}^{\frac{b(N-2)}{2(2-b)}}\phi \left({|y|}^{\frac{b}{2-b}}y\right)dy\\ =\frac{2}{2-b}{\int }_{{\mathbb{R}}^N}{|y|}^{\frac{2b}{2-b}}V\left({|y|}^{\frac{b}{2-b}}y\right)v(y)\psi (y)dy\end{array}$$

and

$$\begin{array}{r}{\int }_{{\mathbb{R}}^N}K(x){|u|}^{p-2}u\phi dx\\ ={\int }_{{\mathbb{R}}^N}K\left({|y|}^{\frac{b}{2-b}}y\right){\left|u\left({|y|}^{\frac{b}{2-b}}y\right)\right|}^{p-2}u\left({|y|}^{\frac{b}{2-b}}y\right)\phi \left({|y|}^{\frac{b}{2-b}}y\right)\frac{2}{2-b}{|y|}^{\frac{bN}{2-b}}dy\\ =\frac{2}{2-b}{\int }_{{\mathbb{R}}^N}{|y|}^{\frac{2s}{2-b}}K\left({|y|}^{\frac{b}{2-b}}y\right){|v(y)|}^{p-2}v(y)\psi (y)dy.\end{array}$$

Therefore,

$$\begin{array}{r}{\int }_{{\mathbb{R}}^N}\mathrm{\nabla }u\mathrm{\nabla }\phi dx+{\int }_{{\mathbb{R}}^N}V(x)u\phi dx-{\int }_{{\mathbb{R}}^N}K(x){|u|}^{p-2}u\phi dx\\ =\frac{2}{2-b}({\int }_{{\mathbb{R}}^N}\sum _{i,j=1}^N{A}_{ij}(y)\frac{\partial v}{\partial y_i}\frac{\partial \psi }{\partial y_j}dy-C_b{\int }_{{\mathbb{R}}^N}\frac{v\psi }{{|y|}^2}dy\\ +{\int }_{{\mathbb{R}}^N}{|y|}^{\frac{2b}{2-b}}V\left({|y|}^{\frac{b}{2-b}}y\right)v(y)\psi (y)dy\\ -{\int }_{{\mathbb{R}}^N}{|y|}^{\frac{2s}{2-b}}K\left({|y|}^{\frac{b}{2-b}}y\right){|v(y)|}^{p-2}v(y)\psi (y)dy)\\ =\frac{2}{2-b}({\int }_{{\mathbb{R}}^N}\sum _{i,j=1}^N{A}_{ij}(y)\frac{\partial v}{\partial y_i}\frac{\partial \psi }{\partial y_j}dy+{\int }_{{\mathbb{R}}^N}{V}_{\ast }(y)v\psi dy-{\int }_{{\mathbb{R}}^N}{K}_{\ast }(y){|v|}^{p-2}v\psi dy)\\ =0.\end{array}$$

This completes the proof. □

This theorem implies that the problem of looking for solutions of (1.1) can be reduced to a problem of looking for solutions of (2.9).

3 The variational functional for Eq. (2.9)

The following inequality is a variant Hardy inequality.

Lemma 3.1 If $v\in H^1({\mathbb{R}}^N)$, then

$${\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }v|}^2}{{|x|}^2}dx\ge \frac{{(N-2)}^2}{4}{\int }_{{\mathbb{R}}^N}\frac{{|v|}^2}{{|x|}^2}dx.$$

(3.1)

Proof We only give the proof of (3.1) for $v\in C_0^{\mathrm{\infty }}({\mathbb{R}}^N)$ since $C_0^{\mathrm{\infty }}({\mathbb{R}}^N)$ is dense in $H^1({\mathbb{R}}^N)$. For $v\in C_0^{\mathrm{\infty }}({\mathbb{R}}^N)$, we have the following identity:

$${|v(x)|}^2=-{\int }_1^{\mathrm{\infty }}\frac{d}{d\lambda }{|v(\lambda x)|}^{2}d\lambda =-2{\int }_1^{\mathrm{\infty }}v(\lambda x)\cdot (x\cdot \mathrm{\nabla }v(\lambda x))d\lambda .$$

By using the Hölder inequality, it follows that

$$\begin{array}{rcl}{\int }_{{\mathbb{R}}^N}\frac{{|v(x)|}^2}{{|x|}^2}dx& =& -2{\int }_1^{\mathrm{\infty }}{\int }_{{\mathbb{R}}^N}\frac{v(\lambda x)}{{|x|}^2}\cdot (x\cdot \mathrm{\nabla }v(\lambda x))dxd\lambda \\ =& -2{\int }_1^{\mathrm{\infty }}\frac{d\lambda }{{\lambda }^{N-1}}{\int }_{{\mathbb{R}}^N}\frac{v(x)}{{|x|}^2}\cdot (x\cdot \mathrm{\nabla }v(x))dx\\ =& -\frac{2}{N-2}{\int }_{{\mathbb{R}}^N}\frac{v(x)}{{|x|}^2}\cdot (x\cdot \mathrm{\nabla }v(x))dx\\ \le & \frac{2}{N-2}{\left({\int }_{{\mathbb{R}}^N}\frac{v^2(x)}{{|x|}^2}dx\right)}^{1/2}{\left({\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }v|}^2}{{|x|}^2}dx\right)}^{1/2}.\end{array}$$

Then we conclude that

$${\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }v|}^2}{{|x|}^2}dx\ge \frac{{(N-2)}^2}{4}{\int }_{{\mathbb{R}}^N}\frac{{|v|}^2}{{|x|}^2}dx.$$

 □

From the definition of $A_{ij}(x)$ (see (2.3)), it is easy to verify that for $u\in H^1({\mathbb{R}}^N)$,

$${\int }_{{\mathbb{R}}^N}\sum _{i,j=1}^N{A}_{ij}(x)\frac{\partial u}{\partial x_i}\frac{\partial u}{\partial x_j}dx={\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }u|}^2}{{|x|}^2}dx.$$

(3.2)

Lemma 3.2 There exist constants $C_1>0$ and $C_2>0$ such that for every $u\in H^1({\mathbb{R}}^N)$,

$$\begin{array}{rl}{C}_1{\parallel u\parallel }^2& \le {\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }u|}^2}{{|x|}^2}dx+{\int }_{{\mathbb{R}}^N}{V}_{\ast }(x){|u|}^{2}dx\\ \le C_2{\parallel u\parallel }^2.\end{array}$$

Proof From conditions (A₁) and (A₂), we deduce that there exists a constant $C>0$ such that

$${|x|}^{\frac{2b}{2-b}}V\left({|x|}^{\frac{b}{2-b}}x\right)\le C(1+{|x|}^{-2}),\mathrm{\forall }x\in {\mathbb{R}}^N\setminus \{0\}.$$

(3.3)

Since

$${\int }_{{\mathbb{R}}^N}{V}_{\ast }(x){|u|}^{2}dx={\int }_{{\mathbb{R}}^N}{|x|}^{\frac{2b}{2-b}}V\left({|x|}^{\frac{b}{2-b}}x\right){|u|}^{2}dx+C_b{\int }_{{\mathbb{R}}^N}\frac{{|u|}^2}{{|x|}^2}dx,$$

by (3.3) and the classical Hardy inequality (see, e.g., [13])

$$\frac{{(N-2)}^2}{4}{\int }_{{\mathbb{R}}^N}\frac{{|u|}^2}{{|x|}^2}dx\le {\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx,\mathrm{\forall }u\in H^1\left({\mathbb{R}}^N\right),$$

we deduce that there exists a constant $C>0$ such that

$${\int }_{{\mathbb{R}}^N}{V}_{\ast }(x){|u|}^{2}dx\le C{\parallel u\parallel }^2.$$

This together with the fact that ${\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }u|}^2}{{|x|}^2}dx\le {\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx$ yields that there exists a constant $C_2>0$ such that

$$\begin{array}{r}{\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }u|}^2}{{|x|}^2}dx+{\int }_{{\mathbb{R}}^N}{V}_{\ast }(x){|u|}^{2}dx\\ \le C_2{\parallel u\parallel }^2,\mathrm{\forall }u\in H^1\left({\mathbb{R}}^N\right).\end{array}$$

(3.4)

If $0<b<2$, then $\frac{b^2}{4}-b<0$ and

$$\begin{array}{rcl}{\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }u|}^2}{{|x|}^2}dx& \ge & {\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx\\ =& {(1-b/2)}^2{\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx.\end{array}$$

(3.5)

In this case, $C_b=\frac{b}{4}(1-\frac{b}{4}){(N-2)}^2>0$ and

$$\begin{array}{rcl}{\int }_{{\mathbb{R}}^N}{V}_{\ast }(x){|u|}^{2}dx& =& {\int }_{{\mathbb{R}}^N}{|x|}^{\frac{2b}{2-b}}V\left({|x|}^{\frac{b}{2-b}}x\right){|u|}^{2}dx+C_b{\int }_{{\mathbb{R}}^N}\frac{{|u|}^2}{{|x|}^2}dx\\ \ge & {\int }_{{\mathbb{R}}^N}{|x|}^{\frac{2b}{2-b}}V\left({|x|}^{\frac{b}{2-b}}x\right){|u|}^{2}dx.\end{array}$$

(3.6)

Conditions (A₁) and (A₂) imply that there exists a constant $C>0$ such that

$${\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+{\int }_{{\mathbb{R}}^N}{|x|}^{\frac{2b}{2-b}}V\left({|x|}^{\frac{b}{2-b}}x\right)u^{2}dx\ge C{\int }_{{\mathbb{R}}^N}{u}^{2}dx.$$

(3.7)

Combining (3.5)-(3.7) yields that there exists a constant $C_1>0$ such that

$$\begin{array}{r}{\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }u|}^2}{{|x|}^2}dx+{\int }_{{\mathbb{R}}^N}{V}_{\ast }(x){|u|}^{2}dx\\ \ge C_1{\parallel u\parallel }^2,\mathrm{\forall }u\in H^1\left({\mathbb{R}}^N\right).\end{array}$$

(3.8)

If $b<0$, (3.7) still holds. From Lemma 3.1 and (3.7), we deduce that there exists a constant $C_1>0$ such that for every $u\in H^1({\mathbb{R}}^N)$,

$$\begin{array}{r}{\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }u|}^2}{{|x|}^2}dx+{\int }_{{\mathbb{R}}^N}{V}_{\ast }(x){|u|}^{2}dx\\ ={\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b)({\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }u|}^2}{{|x|}^2}dx-\frac{{(N-2)}^2}{4}{\int }_{{\mathbb{R}}^N}\frac{{|u|}^2}{{|x|}^2}dx)\\ +{\int }_{{\mathbb{R}}^N}{|x|}^{\frac{2b}{2-b}}V\left({|x|}^{\frac{b}{2-b}}x\right){|u|}^{2}dx\\ \ge {\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+{\int }_{{\mathbb{R}}^N}{|x|}^{\frac{2b}{2-b}}V\left({|x|}^{\frac{b}{2-b}}x\right){|u|}^{2}dx\ge C_1{\parallel u\parallel }^2.\end{array}$$

(3.9)

Then the desired result of this lemma follows from (3.4), (3.8) and (3.9) immediately. □

This lemma implies that

$${\parallel u\parallel }_A={({\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}\frac{{|x\cdot \mathrm{\nabla }u|}^2}{{|x|}^2}dx+{\int }_{{\mathbb{R}}^N}{V}_{\ast }(x){|u|}^{2}dx)}^{1/2}$$

(3.10)

is equivalent to the standard norm $\parallel \cdot \parallel$ in $H^1({\mathbb{R}}^N)$. We denote the inner product associated with ${\parallel \cdot \parallel }_A$ by ${(\cdot ,\cdot )}_A$, i.e.,

$$\begin{array}{rcl}{(u,v)}_A& =& {\int }_{{\mathbb{R}}^N}\mathrm{\nabla }u\mathrm{\nabla }vdx+{\int }_{{\mathbb{R}}^N}{V}_{\ast }(x)uvdx\\ +(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}\frac{(x\cdot \mathrm{\nabla }u)(x\cdot \mathrm{\nabla }v)}{{|x|}^2}dx.\end{array}$$

(3.11)

By the Sobolev inequality, we have

$$S_A:=\underset{u\in H^1({\mathbb{R}}^N)\setminus \{0\}}{inf}\frac{{\parallel u\parallel }_A^2}{{({\int }_{{\mathbb{R}}^N}{|u|}^{p}dx)}^{2/p}}>0$$

(3.12)

and

$${\parallel u\parallel }_A\ge S_A^{\frac{1}{2}}{\left({\int }_{{\mathbb{R}}^N}{|u|}^{p}dx\right)}^{1/p},\mathrm{\forall }u\in H^1\left({\mathbb{R}}^N\right).$$

(3.13)

By conditions (A₁) and (A₂), if $0<b<2$, then $K_{\ast }$ is bounded in ${\mathbb{R}}^N$. Therefore, by (3.13), there exists $C>0$ such that

$${({\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(u^{+}\right)}^{p}dx)}^{1/p}\le C{\parallel u\parallel }_A,\mathrm{\forall }u\in H^1\left({\mathbb{R}}^N\right).$$

(3.14)

However, if $b<0$, $K_{\ast }$ has a singularity at $x=0$, i.e.,

$$K_{\ast }(x)\sim {|x|}^{\frac{2s}{2-b}}K(0),\text{as }|x|\to 0.$$

(3.15)

Recall that $p=2(N-2s/b)/(N-2)$ and $2s/(2-b)>-2s/b$ if $b<0$. Then, by the Hardy-Sobolev inequality (see, for example, [[14], Lemma 3.2]), we deduce that there exists $C>0$ such that (3.14) still holds. Therefore, the functional

$$J(u)=\frac{1}{2}{\parallel u\parallel }_A^2-\frac{1}{p}{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(u^{+}\right)}^{p}dx,u\in H^1\left({\mathbb{R}}^N\right)$$

(3.16)

is a $C^2$ functional defined in $H^1({\mathbb{R}}^N)$. Moreover, it is easy to check that the Gateaux derivative of J is

$$〈J^{\prime }(u),h〉={(u,h)}_A-{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(u^{+}\right)}^{p-1}hdx,\mathrm{\forall }u,h\in H^1\left({\mathbb{R}}^N\right)$$

and the critical points of J are nonnegative solutions of (2.9).

4 Some minimizing problems

For $\theta =({\theta }_1,\dots ,{\theta }_N)\in {\mathbb{R}}^N$ with $|\theta |=1$, let

$$B_{ij}(\theta )={\delta }_{ij}+(\frac{b^2}{4}-b){\theta }_i{\theta }_j,i,j=1,\dots ,N.$$

(4.1)

By this definition, we have, for $u\in H^1({\mathbb{R}}^N)$,

$${\int }_{{\mathbb{R}}^N}\sum _{i,j=1}^N{B}_{ij}(\theta )\frac{\partial u}{\partial x_i}\frac{\partial u}{\partial x_j}dx={\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}{|\theta \cdot \mathrm{\nabla }u|}^{2}dx.$$

(4.2)

From

$$\begin{array}{rcl}(1+|\frac{b^2}{4}-b|){\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx& \ge & {\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}{|\theta \cdot \mathrm{\nabla }u|}^{2}dx\\ \ge & \{\begin{array}{cc}{(1-b/2)}^2{\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx,\hfill & 0<b<2,\hfill \\ {\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx,\hfill & b<0,\hfill \end{array}\end{array}$$

we deduce that the norm defined by

$${\parallel u\parallel }_{\theta }:={({\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u|}^{2}dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}{|\theta \cdot \mathrm{\nabla }u|}^{2}dx+a{\int }_{{\mathbb{R}}^N}{|u|}^{2}dx)}^{1/2}$$

(4.3)

is equivalent to the standard norm $\parallel \cdot \parallel$ in $H^1({\mathbb{R}}^N)$. The inner product corresponding to ${\parallel \cdot \parallel }_{\theta }$ is

$${(u,v)}_{\theta }={\int }_{{\mathbb{R}}^N}\mathrm{\nabla }u\mathrm{\nabla }vdx+a{\int }_{{\mathbb{R}}^N}uvdx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}(\theta \cdot \mathrm{\nabla }u)(\theta \cdot \mathrm{\nabla }v)dx.$$

Lemma 4.1 The infimum

$$\underset{u\in H^1({\mathbb{R}}^N)\setminus \{0\}}{inf}\frac{{\parallel u\parallel }_{\theta }^2}{{({\int }_{{\mathbb{R}}^N}{|u|}^{p}dx)}^{2/p}}$$

(4.4)

is independent of $\theta \in {\mathbb{R}}^N$ with $|\theta |=1$.

Proof In this proof, we always view a vector in ${\mathbb{R}}^N$ as a $1\times N$ matrix, and we use $A^T$ to denote the conjugate matrix of a matrix A.

For any $\theta ,{\theta }^{\prime }\in {\mathbb{R}}^N$ with $|\theta |=|{\theta }^{\prime }|=1$, let G be an $N\times N$ orthogonal matrix such that ${\theta }^{\prime }\cdot G^T=\theta$. For any $u\in H^1({\mathbb{R}}^N)$, let $v(x)=u(xG)$, $x\in {\mathbb{R}}^N$. The assumption that G is an $N\times N$ orthogonal matrix implies that $G{G}^T=I$, where I is the $N\times N$ identity matrix. Then it is easy to check that

$${\int }_{{\mathbb{R}}^N}{|v|}^{2}dx={\int }_{{\mathbb{R}}^N}{|u|}^{2}dx,{\int }_{{\mathbb{R}}^N}{|v|}^{p}dx={\int }_{{\mathbb{R}}^N}{|u|}^{p}dx.$$

(4.5)

Note that

$$\mathrm{\nabla }v(x)=(\mathrm{\nabla }u)(xG)\cdot G.$$

(4.6)

By $G{G}^T=I$, we have

$$\begin{array}{rcl}{|\mathrm{\nabla }v(x)|}^2& =& \mathrm{\nabla }v(x)\cdot {(\mathrm{\nabla }v(x))}^T\\ =& (\mathrm{\nabla }u)(xG)\cdot G\cdot G^T\cdot {((\mathrm{\nabla }u)(xG))}^T={|(\mathrm{\nabla }u)(xG)|}^2.\end{array}$$

It follows that

$${\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }v(x)|}^{2}dx={\int }_{{\mathbb{R}}^N}{|(\mathrm{\nabla }u)(xG)|}^{2}dx={\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }u(x)|}^{2}dx.$$

(4.7)

By (4.6) and ${\theta }^{\prime }\cdot G^T=\theta$, we get that

$$\begin{array}{rcl}\sum _{i=1}^N{\theta }_i^{\prime }\frac{\partial v}{\partial x_i}& =& {\theta }^{\prime }\cdot {((\mathrm{\nabla }u)(xG)\cdot G)}^T={\theta }^{\prime }\cdot G^T\cdot {((\mathrm{\nabla }u)(xG))}^T=\theta \cdot {((\mathrm{\nabla }u)(xG))}^T\\ =& \sum _{i=1}^N{\theta }_i\left(\frac{\partial u}{\partial y_i}\right)(xG).\end{array}$$

It follows that

$$\begin{array}{rcl}{\int }_{{\mathbb{R}}^N}{|{\theta }^{\prime }\cdot \mathrm{\nabla }v|}^{2}dx& =& {\int }_{{\mathbb{R}}^N}{\left|\sum _{i=1}^N{\theta }_i^{\prime }\frac{\partial v}{\partial x_i}\right|}^{2}dx\\ =& {\int }_{{\mathbb{R}}^N}{|\sum _{i=1}^N{\theta }_i\left(\frac{\partial u}{\partial y_i}\right)(xG)|}^{2}dx\\ =& {\int }_{{\mathbb{R}}^N}{\left|\sum _{i=1}^N{\theta }_i\frac{\partial u}{\partial x_i}\right|}^{2}dx={\int }_{{\mathbb{R}}^N}{|\theta \cdot \mathrm{\nabla }u|}^{2}dx.\end{array}$$

(4.8)

By (4.5), (4.7) and (4.8), we get that ${\parallel v\parallel }_{{\theta }^{\prime }}^2={\parallel u\parallel }_{\theta }^2$. This together with (4.5) leads to the result of this lemma. □

Since the infimum (4.4) is independent of $\theta \in {\mathbb{R}}^N$ with $|\theta |=1$, we denote it by S.

Lemma 4.2 Let $S_p$ be the infimum in (1.9). Then $S={(1-b/2)}^{\frac{p-2}{p}}{S}_p$.

Proof Choosing $\theta =(1,0,\dots ,0)$ in ${\parallel \cdot \parallel }_{\theta }$, we have

$${\parallel u\parallel }_{\theta }^2={(1-\frac{b}{2})}^2{\int }_{{\mathbb{R}}^N}{\left|\frac{\partial u}{\partial x_1}\right|}^{2}dx+\sum _{i=2}^N{\int }_{{\mathbb{R}}^N}{\left|\frac{\partial u}{\partial x_i}\right|}^{2}dx+a{\int }_{{\mathbb{R}}^N}{u}^{2}dx.$$

By Lemma 4.1, we have

$$S=\underset{u\in H^1({\mathbb{R}}^N)\setminus \{0\}}{inf}\frac{{(1-\frac{b}{2})}^2{\int }_{{\mathbb{R}}^N}{|\frac{\partial u}{\partial x_1}|}^{2}dx+{\sum }_{i=2}^N{\int }_{{\mathbb{R}}^N}{|\frac{\partial u}{\partial x_i}|}^{2}dx+a{\int }_{{\mathbb{R}}^N}{u}^{2}dx}{{({\int }_{{\mathbb{R}}^N}{|u|}^{p}dx)}^{2/p}}.$$

Let

$$v(x)=u((1-b/2)x_1,x_2,\dots ,x_N),x\in {\mathbb{R}}^N.$$

Then

$$\begin{array}{r}\frac{{(1-\frac{b}{2})}^2{\int }_{{\mathbb{R}}^N}{|\frac{\partial u}{\partial x_1}|}^{2}dx+{\sum }_{i=2}^N{\int }_{{\mathbb{R}}^N}{|\frac{\partial u}{\partial x_i}|}^{2}dx+a{\int }_{{\mathbb{R}}^N}{u}^{2}dx}{{({\int }_{{\mathbb{R}}^N}{|u|}^{p}dx)}^{2/p}}\\ ={(1-b/2)}^{\frac{p-2}{p}}\frac{{\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }v|}^{2}dx+a{\int }_{{\mathbb{R}}^N}{v}^{2}dx}{{({\int }_{{\mathbb{R}}^N}{|v|}^{p}dx)}^{2/p}}.\end{array}$$

It follows that

$$S={(1-b/2)}^{\frac{p-2}{p}}\underset{v\in H^1({\mathbb{R}}^N)\setminus \{0\}}{inf}\frac{{\int }_{{\mathbb{R}}^N}{|\mathrm{\nabla }v|}^{2}dx+a{\int }_{{\mathbb{R}}^N}{v}^{2}dx}{{({\int }_{{\mathbb{R}}^N}{|v|}^{p}dx)}^{2/p}}={(1-b/2)}^{\frac{p-2}{p}}{S}_p.$$

 □

Since the functionals ${\parallel u\parallel }_{\theta }^2$ and ${\int }_{{\mathbb{R}}^N}{|u|}^{p}dx$ are invariant by translations, the same argument as the proof of [[11], Theorem 1.34] yields that there exists a positive minimizer $U_{\theta }$ for the infimum S. Moreover, from the Lagrange multiplier rule, it is a solution of

$$-\sum _{i,j=1}^N\frac{\partial }{\partial y_j}(B_{ij}(\theta )\frac{\partial u}{\partial y_i})+au=S{\left(u^{+}\right)}^{p-1}\text{in }{\mathbb{R}}^N,$$

and ${(\mu /S)}^{-1/(p-2)}{U}_{\theta }$ is a solution of

$$-\sum _{i,j=1}^N\frac{\partial }{\partial y_j}(B_{ij}(\theta )\frac{\partial u}{\partial y_i})+au=\mu {\left(u^{+}\right)}^{p-1}\text{in }{\mathbb{R}}^N.$$

(4.9)

In the next section, we shall show that Eq. (4.9) is the ‘limit’ equation of

$$-\sum _{i,j=1}^N\frac{\partial }{\partial y_j}(A_{ij}(x)\frac{\partial u}{\partial y_i})+V_{\ast }(x)u=K_{\ast }(x){\left(u^{+}\right)}^{p-1}\text{in }{\mathbb{R}}^N.$$

(4.10)

It is easy to verify that

$$J_{\theta }(u)=\frac{1}{2}{\parallel u\parallel }_{\theta }^2-\frac{\mu }{p}{\int }_{{\mathbb{R}}^N}{\left(u^{+}\right)}^{p}dx,u\in H^1\left({\mathbb{R}}^N\right)$$

(4.11)

is a $C^2$ functional defined in $H^1({\mathbb{R}}^N)$, the Gateaux derivative of $J_{\theta }$ is

$$〈J_{\theta }^{\prime }(u),h〉={(u,h)}_{\theta }-\mu {\int }_{{\mathbb{R}}^N}{\left(u^{+}\right)}^{p-1}hdx,\mathrm{\forall }u,h\in H^1\left({\mathbb{R}}^N\right),$$

and the critical points of this functional are solutions of (4.9).

Lemma 4.3 Let $\theta \in {\mathbb{R}}^N$ satisfy $|\theta |=1$. If $u\ne 0$ is a critical point of $J_{\theta }$, then

$$J_{\theta }(u)\ge (\frac{1}{2}-\frac{1}{p}){\mu }^{-\frac{2}{p-2}}{S}^{\frac{p}{p-2}}.$$

(4.12)

Proof Since u is a critical point of $J_{\theta }$, we have

$$0=〈J_{\theta }^{\prime }(u),u〉={\parallel u\parallel }_{\theta }^2-\mu {\int }_{{\mathbb{R}}^N}{\left(u^{+}\right)}^{p}dx.$$

(4.13)

It follows that

$$J_{\theta }(u)=(\frac{1}{2}-\frac{1}{p})\mu {\int }_{{\mathbb{R}}^N}{\left(u^{+}\right)}^{p}dx.$$

(4.14)

Since $u\ne 0$, by ${\parallel u\parallel }_{\theta }^2=\mu {\int }_{{\mathbb{R}}^N}{(u^{+})}^{p}dx$ and ${\parallel u\parallel }_{\theta }^2\ge S{({\int }_{{\mathbb{R}}^N}{(u^{+})}^{p}dx)}^{2/p}$, we get that

$${\int }_{{\mathbb{R}}^N}{\left(u^{+}\right)}^{p}dx\ge {(S/\mu )}^{p/(p-2)}.$$

This together with (4.14) yields the result of this lemma. □

5 The Palais-Smale condition for the functional J

Recall that J is the functional defined by (3.16). By a ${(PS)}_c$ sequence of J, we mean a sequence $\{u_n\}\subset H^1({\mathbb{R}}^N)$ such that $J(u_n)\to c$ and $J^{\prime }(u_n)\to 0$ in $H^{-1}({\mathbb{R}}^N)$ as $n\to \mathrm{\infty }$, where $H^{-1}({\mathbb{R}}^N)$ denotes the dual space of $H^1({\mathbb{R}}^N)$. J is called satisfying the ${(PS)}_c$ condition if every ${(PS)}_c$ sequence of J contains a convergent subsequence in $H^1({\mathbb{R}}^N)$.

Our main result in this section reads as follows.

Theorem 5.1 Under assumptions (A₁) and (A₂), let $\{u_n\}\subset H^1({\mathbb{R}}^N)$ be a ${(PS)}_c$ sequence of J. Then replacing $\{u_n\}$ if necessary by a subsequence, there exist a solution $u_0\in H^1({\mathbb{R}}^N)$ of Eq. (4.10), a finite sequence $\{{\theta }_l\in {\mathbb{R}}^N\mid |{\theta }_l|=1,1\le l\le k\}$, k functions $\{u_l\mid 1\le i\le k\}\subset H^1({\mathbb{R}}^N)$ and k sequences $\{y_n^l\}\subset {\mathbb{R}}^N$ satisfying:

• (i) $-{\sum }_{i,j=1}^N\frac{\partial }{\partial y_j}(B_{ij}({\theta }_l)\frac{\partial u_l}{\partial y_i})+a{u}_l=\mu {(u_l^{+})}^{p-1}$ in ${\mathbb{R}}^N$,

• (ii) $|y_n^l|\to \mathrm{\infty }$, $|y_n^l-y_n^{l^{\prime }}|\to \mathrm{\infty }$, $l\ne l^{\prime }$, $n\to \mathrm{\infty }$,

• (iii) $\parallel u_n-u_0-{\sum }_{l=1}^k{u}_l(\cdot -y_n^l)\parallel \to 0$,

• (iv) $J(u_0)+{\sum }_{i=1}^l{J}_{{\theta }_l}(u_l)=c$.

This theorem gives a precise representation of the ${(PS)}_c$ sequence for the functional J. Through it, partial compactness for J can be regained (see Corollary 5.8).

To prove this theorem, we need some lemmas. Our proof of this theorem is inspired by the proof of [[11], Theorem 8.4].

Lemma 5.2 Let $u\in H^1({\mathbb{R}}^N)$. Then, for any sequence $\{y_n\}\subset {\mathbb{R}}^N$,

$$\underset{R\to \mathrm{\infty }}{lim}\underset{n}{sup}{\int }_{|x|>R}{K}_{\ast }(x+y_n){|u|}^{p}dx=0.$$

If $|y_n|\to \mathrm{\infty }$, $n\to \mathrm{\infty }$, then

$$\underset{n\to \mathrm{\infty }}{lim}{\int }_{{\mathbb{R}}^N}|K_{\ast }(x+y_n)-\mu |\cdot {|u|}^{p}dx=0.$$

Proof If $2>b>0$, then $K_{\ast }$ is bounded in ${\mathbb{R}}^N$. In this case, the result of this lemma is obvious. If $b<0$, then $K_{\ast }(x)\sim {|x|}^{\frac{2s}{2-b}}K(0)$ as $|x|\to 0$. Since $2s/(2-b)>-2s/b$, by Lemma 3.2 of [14], the map $v\mapsto K_{\ast }^{1/p}v$ from $H^1({\mathbb{R}}^N)\to L_{\mathrm{loc}}^p({\mathbb{R}}^N)$ is compact. Therefore, for any $ϵ>0$, there exists ${\delta }_{ϵ}>0$ such that

$$\underset{n}{sup}{\int }_{|x|\le {\delta }_{ϵ}}{K}_{\ast }(x){|u(x-y_n)|}^{p}dx\le ϵ.$$

And there exists $D(ϵ)>0$ depending only on ϵ such that $K_{\ast }(x)\le D(ϵ)$, $|x|\ge {\delta }_{ϵ}$. Then, for every n,

$$\begin{array}{r}{\int }_{|x|>R}{K}_{\ast }(x+y_n){|u|}^{p}dx\\ \le {\int }_{\{x||x+y_n|\le {\delta }_{ϵ},|x|>R\}}{K}_{\ast }(x+y_n){|u|}^{p}dx+{\int }_{\{x\mid |x+y_n|>{\delta }_{ϵ},|x|>R\}}{K}_{\ast }(x+y_n){|u|}^{p}dx\\ \le ϵ+C(ϵ){\int }_{|x|>R}{|u|}^{p}dx.\end{array}$$

It follows that ${lim\hspace{0.17em}sup}_{R\to \mathrm{\infty }}{sup}_n{\int }_{|x|>R}{K}_{\ast }(x+y_n){|u|}^{p}dx\le ϵ$. Now let $ϵ\to 0$.

Using the same argument as above, for any $ϵ>0$, there exist ${\delta }_{ϵ}$ and $D(ϵ)$ such that

$$\underset{n}{sup}{\int }_{|x+y_n|\le {\delta }_{ϵ}}|K_{\ast }(x+y_n)-\mu |\cdot {|u|}^{p}dx\le ϵ$$

and

$$|K_{\ast }(x+y_n)-\mu |\cdot {|u|}^{p}dx\le (D(ϵ)+\mu ){|u|}^p,|x+y_n|\ge {\delta }_{ϵ}.$$

Since $y_n\to \mathrm{\infty }$, we have $lim{K}_{\ast }(x+y_n)=\mu$. Then, using the Lebesgue theorem and the above two inequalities, we get that

$$\underset{n\to \mathrm{\infty }}{lim\hspace{0.17em}sup}{\int }_{{\mathbb{R}}^N}|K_{\ast }(x+y_n)-\mu |\cdot {|u|}^{p}dx\le ϵ.$$

Let $ϵ\to 0$. Then we get the desired result of this lemma. □

Lemma 5.3 Let $\rho >0$. If $\{u_n\}$ is bounded in $H^1({\mathbb{R}}^N)$ and

$$\underset{y\in {\mathbb{R}}^N}{sup}{\int }_{B(y,\rho )}{|u_n|}^{2}dx\to 0,n\to \mathrm{\infty },$$

(5.1)

then $K_{\ast }^{1/p}{u}_n\to 0$ in $L^p({\mathbb{R}}^N)$.

Proof Since $2s/(2-b)>-2s/b$, by Lemma 3.2 of [14], the map $v\mapsto K_{\ast }^{1/p}v$ from $H^1({\mathbb{R}}^N)\to L_{\mathrm{loc}}^p({\mathbb{R}}^N)$ is compact. Therefore, for any $ϵ>0$, there exists ${\delta }_{ϵ}>0$ such that

$$\underset{n}{sup}{\int }_{|x|\le {\delta }_{ϵ}}{K}_{\ast }(x){|u_n|}^{p}dx\le ϵ.$$

And there exists $D(ϵ)>0$ depending only on ϵ such that $K_{\ast }(x)\le D(ϵ)$, $|x|\ge {\delta }_{ϵ}$. By (5.1) and the Lions lemma (see, for example, [[11], Lemma 1.21]), we get that

$${\int }_{|x|\ge {\delta }_{ϵ}}{K}_{\ast }(x){|u_n|}^{p}dx\le D(ϵ){\int }_{{\mathbb{R}}^N}{|u_n|}^{p}dx\to 0,n\to \mathrm{\infty }.$$

Therefore, ${lim\hspace{0.17em}sup}_{n\to \mathrm{\infty }}{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){|u_n|}^{p}dx\le ϵ$. Now let $ϵ\to 0$. □

Lemma 5.4 Let $\{y_n\}\subset {\mathbb{R}}^N$. If $u_n\rightharpoonup u$ in $H^1({\mathbb{R}}^N)$, then

$$K_{\ast }(x+y_n){\left(u_n^{+}\right)}^{p-1}-K_{\ast }(x+y_n){\left({(u_n-u)}^{+}\right)}^{p-1}-K_{\ast }(x+y_n){\left(u^{+}\right)}^{p-1}\to 0\mathit{\text{in }}{H}^{-1}\left({\mathbb{R}}^N\right).$$

One can follow the proof of [[11], Lemma 8.1] step by step and use Lemma 5.2 to give the proof of this lemma.

The following lemma is a variant Brézis-Lieb lemma (see [15]) and its proof is similar to that of [[11], Lemma 1.32].

Lemma 5.5 Let $\{u_n\}\subset H^1({\mathbb{R}}^N)$ and $\{y_n\}\subset {\mathbb{R}}^N$. If

• (a) $\{u_n\}$ is bounded in $H^1({\mathbb{R}}^N)$,

• (b) $u_n\to u$ a.e. on ${\mathbb{R}}^N$, then

$$\underset{n\to \mathrm{\infty }}{lim}{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x+y_n)\cdot |{\left(u_n^{+}\right)}^p-{\left({(u_n-u)}^{+}\right)}^p-{\left(u^{+}\right)}^p|dx=0.$$

Proof Let

$$j(t)=\{\begin{array}{cc}{t}^p,\hfill & t\ge 0,\hfill \\ 0,\hfill & t<0.\hfill \end{array}$$

Then j is a convex function. From [[15], Lemma 3], we have that for any $ϵ>0$, there exists $C(ϵ)>0$ such that for all $a,b\in \mathbb{R}$,

$$|j(a+b)-j(b)|\le ϵj(a)+C(ϵ)j(b).$$

(5.2)

Hence

$$\begin{array}{rcl}{f}_n^{ϵ}& :=& {(K_{\ast }(x+y_n)\cdot |{\left(u_n^{+}\right)}^p-{\left({(u_n-u)}^{+}\right)}^p-{\left(u^{+}\right)}^p|-ϵK_{\ast }(x+y_n)\cdot {\left({(u_n-u)}^{+}\right)}^p)}^{+}\\ \le & (1+C(ϵ))K_{\ast }(x+y_n)\cdot {\left(u^{+}\right)}^p.\end{array}$$

By Lemma 3.2 of [14], the map $v\mapsto K_{\ast }^{1/p}v$ from $H^1({\mathbb{R}}^N)\to L_{\mathrm{loc}}^p({\mathbb{R}}^N)$ is compact. We get that there exists ${\delta }_{ϵ}>0$ such that for any n,

$${\int }_{|x+y_n|<{\delta }_{ϵ}}{f}_n^{ϵ}dx<ϵ.$$

(5.3)

And there exists $D(ϵ)>0$ depending only on ϵ such that $K_{\ast }(x)\le D(ϵ)$, $|x|\ge {\delta }_{ϵ}$. Then

$$f_n^{ϵ}\le (1+C(ϵ))D(ϵ)\cdot {\left(u^{+}\right)}^p,|x+y_n|\ge {\delta }_{ϵ}.$$

By the Lebesgue theorem, ${\int }_{|x+y_n|\ge {\delta }_{ϵ}}{f}_n^{ϵ}dx\to 0$, $n\to \mathrm{\infty }$. This together with (5.3) yields

$$\underset{n\to \mathrm{\infty }}{lim\hspace{0.17em}sup}{\int }_{{\mathbb{R}}^N}{f}_n^{ϵ}dx\le ϵ.$$

The left proof is the same as the proof of [[11], Lemma 1.32]. □

Lemma 5.6 If

$$\begin{array}{c}{u}_n\rightharpoonup u\mathit{\text{in }}{H}^1\left({\mathbb{R}}^N\right),\hfill \\ u_n\to u\mathit{\text{a.e. on }}{\mathbb{R}}^N,\hfill \\ J(u_n)\to c,\hfill \\ J^{\prime }(u_n)\to 0\mathit{\text{in }}{H}^{-1}\left({\mathbb{R}}^N\right),\hfill \end{array}$$

then $J^{\prime }(u)=0$ in $H^{-1}({\mathbb{R}}^N)$ and $v_n:=u_n-u$ is such that

$$\begin{array}{c}{\parallel v_n\parallel }_A^2={\parallel u_n\parallel }_A^2-{\parallel u\parallel }_A^2+o(1),\hfill \\ J(v_n)\to c-J(u),\hfill \\ J^{\prime }(v_n)\to 0\mathit{\text{in }}{H}^{-1}\left({\mathbb{R}}^N\right).\hfill \end{array}$$

Proof (1) Since $u_n\rightharpoonup u$ in $H^1({\mathbb{R}}^N)$, we get that as $n\to \mathrm{\infty }$,

$${\parallel v_n\parallel }_A^2-{\parallel u_n\parallel }_A^2={(u_n-u,u_n-u)}_A-{\parallel u_n\parallel }_A^2={\parallel u\parallel }_A^2-2{(u_n,u)}_A\to -{\parallel u\parallel }_A^2.$$

Therefore,

$${\parallel v_n\parallel }_A^2={\parallel u_n\parallel }_A^2-{\parallel u\parallel }_A^2+o(1).$$

(5.4)

• (2) Lemma 5.5 implies

  $${\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(v_n^{+}\right)}^{p}dx={\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(u_n^{+}\right)}^{p}dx-{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(u^{+}\right)}^{p}dx+o(1).$$

  (5.5)

By (5.4), (5.5) and the assumption $J(u_n)\to c$, we get that

$$J(v_n)\to c-J(u),n\to \mathrm{\infty }.$$

• (3) Since $J^{\prime }(u_n)\to 0$ in $H^{-1}({\mathbb{R}}^N)$ and $u_n\rightharpoonup u$, it is easy to verify that $J^{\prime }(u)=0$. For $h\in H^1({\mathbb{R}}^N)$,

  $$\begin{array}{rcl}〈J^{\prime }(v_n),h〉& =& {(v_n,h)}_A-{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(v_n^{+}\right)}^{p-1}hdx\\ =& {(u_n,h)}_A-{(u,h)}_A-{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(v_n^{+}\right)}^{p-1}hdx.\end{array}$$

  (5.6)

By Lemma 5.4, we have

$$\begin{array}{r}\underset{\parallel h\parallel \le 1}{sup}|{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(v_n^{+}\right)}^{p-1}hdx-{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(u_n^{+}\right)}^{p-1}hdx+{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(u^{+}\right)}^{p-1}hdx|\\ \to 0,n\to \mathrm{\infty }.\end{array}$$

(5.7)

Combining (5.6) and (5.7) leads to $J^{\prime }(v_n)=J^{\prime }(u_n)-J^{\prime }(u)+o(1)$. Then, by $J^{\prime }(u_n)\to 0$ in $H^{-1}({\mathbb{R}}^N)$ and $J^{\prime }(u)=0$, we obtain that $J^{\prime }(v_n)\to 0$ in $H^{-1}({\mathbb{R}}^N)$. □

Lemma 5.7 If $|y_n|\to \mathrm{\infty }$ and as $n\to \mathrm{\infty }$,

$$\begin{array}{c}{u}_n(\cdot +y_n)\rightharpoonup u\mathit{\text{in }}{H}^1\left({\mathbb{R}}^N\right),\hfill \\ u_n(\cdot +y_n)\to u\mathit{\text{a.e. on }}{\mathbb{R}}^N,\hfill \\ J(u_n)\to c,\hfill \\ J^{\prime }(u_n)\to 0\mathit{\text{in }}{H}^{-1}\left({\mathbb{R}}^N\right),\hfill \end{array}$$

then there exists $\theta \in {\mathbb{R}}^N$ with $|\theta |=1$ such that $J_{\theta }^{\prime }(u)=0$ and $v_n=u_n-u(\cdot -y_n)$ is such that

$$\begin{array}{c}{\parallel v_n\parallel }^2={\parallel u_n\parallel }^2-{\parallel u\parallel }^2+o(1),\hfill \\ J(v_n)\to c-J_{\theta }(u),\hfill \\ J^{\prime }(v_n)\to 0\mathit{\text{in }}{H}^{-1}\left({\mathbb{R}}^N\right).\hfill \end{array}$$

Proof We divide the proof into several steps.

• (1) Since $u_n(\cdot +y_n)\rightharpoonup u$ in $H^1({\mathbb{R}}^N)$, it is clear that

  $${\parallel v_n\parallel }^2={\parallel v_n(\cdot +y_n)\parallel }^2={\parallel u_n(\cdot +y_n)\parallel }^2+{\parallel u\parallel }^2-2(u_n(\cdot +y_n),u)={\parallel u_n\parallel }^2-{\parallel u\parallel }^2+o(1).$$

• (2) For any $h\in H^1({\mathbb{R}}^N)$,

  $$〈J^{\prime }(u_n),h(\cdot -y_n)〉={(u_n,h(\cdot -y_n))}_A-{\int }_{{\mathbb{R}}^N}{K}_{\ast }(x){\left(u_n^{+}\right)}^{p-1}h(\cdot -y_n)dx.$$

  (5.8)

By the definition of the inner product ${(\cdot ,\cdot )}_A$ (see (3.11)), we have

$$\begin{array}{r}{(u_n,h(\cdot -y_n))}_A\\ ={\int }_{{\mathbb{R}}^N}\mathrm{\nabla }{u}_n\mathrm{\nabla }h(\cdot -y_n)dx+(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}\frac{(x\cdot \mathrm{\nabla }{u}_n)(x\cdot \mathrm{\nabla }h(\cdot -y_n))}{{|x|}^2}dx\\ +{\int }_{{\mathbb{R}}^N}{V}_{\ast }(x)u_{n}h(\cdot -y_n)dx\\ ={\int }_{{\mathbb{R}}^N}\mathrm{\nabla }{u}_n(\cdot +y_n)\mathrm{\nabla }hdx+a{\int }_{{\mathbb{R}}^N}{u}_n(\cdot +y_n)\cdot hdx\\ +{\int }_{{\mathbb{R}}^N}(V_{\ast }(x+y_n)-a)u_n(\cdot +y_n)\cdot hdx\\ +(\frac{b^2}{4}-b){\int }_{{\mathbb{R}}^N}(\frac{\frac{x}{|y_n|}+\frac{y_n}{|y_n|}}{|\frac{x}{|y_n|}+\frac{y_n}{|y_n|}|}\cdot \mathrm{\nabla }{u}_n(\cdot +y_n))(\frac{\frac{x}{|y_n|}+\frac{y_n}{|y_n|}}{|\frac{x}{|y_n|}+\frac{y_n}{|y_n|}|}\cdot \mathrm{\nabla }h)dx\\ :=I+\mathit{II}+\mathit{III}.\end{array}$$

(5.9)

Since $u_n(\cdot +y_n)\rightharpoonup u$ in $H^1({\mathbb{R}}^N)$, we have

$$\begin{array}{rl}I& ={\int }_{{\mathbb{R}}^N}\mathrm{\nabla }{u}_n(\cdot +y_n)\mathrm{\nabla }hdx+a{\int }_{{\mathbb{R}}^N}{u}_n(\cdot +y_n)\cdot hdx=(u_n(\cdot +y_n),h)\\ \to {\int }_{{\mathbb{R}}^N}\mathrm{\nabla }u\mathrm{\nabla }hdx+a{\int }_{{\mathbb{R}}^N}uhdx,n\to \mathrm{\infty }.\end{array}$$

(5.10)

By assumption (A₂) and the definition of $V_{\ast }$, we have ${lim}_{|x|\to \mathrm{\infty }}{V}_{\ast }(x)=a$. This yields

$$\underset{n}{sup}{\int }_{|x|\ge R}|V_{\ast }(x)-a|\cdot {|h(x-y_n)|}^{2}dx\to 0,R\to \mathrm{\infty }.$$

Moreover, together with (2.8) and the fact that $|y_n|\to \mathrm{\infty }$ yields that for any fixed $R>0$,

$$\begin{array}{r}{\int }_{|x|<R}|V_{\ast }(x)-a|\cdot {|h(\cdot -y_n)|}^{2}dx\\ \le C({\int }_{|x|<R}{|\mathrm{\nabla }h(\cdot -y_n)|}^{2}dx+{\int }_{|x|<R}{|h(\cdot -y_n)|}^{2}dx)\to 0,n\to \mathrm{\infty }.\end{array}$$

Combining the above two limits leads to

$${\int }_{{\mathbb{R}}^N}|V_{\ast }(x+y_n)-a|\cdot {|h|}^{2}dx\to 0,n\to \mathrm{\infty }.$$

(5.11)

By (5.11) and the Hölder inequality, we have

$$\begin{array}{rcl}|\mathit{II}|& =& |{\int }_{{\mathbb{R}}^N}(V_{\ast }(x+y_n)-a)u_n(\cdot +y_n)\cdot hdx|\\ \le & {({\int }_{{\mathbb{R}}^N}|V_{\ast }(x+y_n)-a|u_n^2(\cdot +y_n)dx)}^{\frac{1}{2}}{\left({\int }_{{\mathbb{R}}^N}|V_{\ast }(x+y_n)-a|h^{2}dx\right)}^{\frac{1}{2}}\\ \le & C{\left({\int }_{{\mathbb{R}}^N}|V_{\ast }(x+y_n)-a|h^{2}dx\right)}^{\frac{1}{2}}\to 0,n\to \mathrm{\infty }.\end{array}$$

(5.12)

Since $\mathrm{\nabla }h\in L^2({\mathbb{R}}^N)$, for any $ϵ>0$, there exists $R_{ϵ}>0$ such that

$${\int }_{{\mathbb{R}}^N\setminus \{|x|<R_{ϵ}\}}{|\mathrm{\nabla }h|}^{2}dx<ϵ.$$

It follows that

$${\int }_{{\mathbb{R}}^N\setminus \{|x|<R_{ϵ}\}}\frac{{|(\frac{x}{|y_n|}+\frac{y_n}{|y_n|})\cdot \mathrm{\nabla }h|}^2}{{|\frac{x}{|y_n|}+\frac{y_n}{|y_n|}|}^2}dx\le {\int }_{{\mathbb{R}}^N\setminus \{|x|<R_{ϵ}\}}{|\mathrm{\nabla }h|}^{2}dx<ϵ.$$

(5.13)

Then

$$\begin{array}{r}|{\int }_{{\mathbb{R}}^N\setminus \{|x|<R_{ϵ}\}}(\frac{\frac{x}{|y_n|}+\frac{y_n}{|y_n|}}{|\frac{x}{|y_n|}+\frac{y_n}{|y_n|}|}\cdot \mathrm{\nabla }{u}_n(\cdot +y_n))(\frac{\frac{x}{|y_n}}{}\end{array}$$