### The key design

The key design of the proposed color image encryption algorithm is shown in Fig. 2. It consists of five parts, chaotic system initial values(*x*_{0},*y*_{0},*z*_{0},*w*_{0}), parameters *a*, *b*, *α*, *β*, *γ*, *q*, *ρ*, cycle numbers *m*, *n*, starting acid base *c*_{0}(*c*_{0}∈A, T, C, G) and DNA encoding rules in Table 1 *α*1, *β*1(*α*1, *β*1∈ [1, 8]).

### Image encryption algorithm

#### Pixel position scrambling

The pixel location is scrambled in order to destroy correlation of the original image, and an image is rearranged, which makes the image become disturbed. The random sequences through the fractional-order 4D hyperchaotic memristive system are generated, and the image is permutated. The detailed confusion process can be presented as the following steps.

*Step* **1.** The input is color original image *I* with the size of *M* × *N* × 3. Setting secret key values *a*, *b*, *α*, *β*, *γ*, *q*, *ρ*, *x*_{0}, *y*_{0}, *z*_{0}, *w*_{0}. New initial conditions of the fractional-order 4D hyperchaotic memristive system are generated by

$$ s=\frac{\left[\sum \limits_{i=1}^M\sum \limits_{j=1}^N\sum \limits_{k=1}^3I\left(i,j,k\right)\right]}{10^{10}} $$

(11)

$$ \left\{\begin{array}{l}{x}_0^{\hbox{'}}={x}_0+s\\ {}{y}_0^{\hbox{'}}={y}_0+s\\ {}{z}_0^{\hbox{'}}={z}_0+s\\ {}{w}_0^{\hbox{'}}={w}_0+s\end{array}\right. $$

(12)

*Step* **2.** Setting the *L* = max(*M*, *N*). Let the chaotic system (9) iterate for (*m* + *L*) times based on new initial conditions, and then throw out the former *m* values to improve initial value sensitivity. The four chaotic sequences {*x*_{i}}*L i = 1*, {*y*_{i}}*L i = 1*, {*z*_{i}}*L i = 1* and {*w*_{i}}*L i = 1* are obtained by Eq. (9). The following shift step numbers are used for scrambling:

$$ Bri=\operatorname{mod}\left(\left\lfloor \left|{x}_i\right|\times {10}^{16}\right\rfloor, \frac{N}{2}\right) $$

(13)

$$ Bcj=\operatorname{mod}\left(\left\lfloor \left|{y}_j\right|\times {10}^{16}\right\rfloor, \frac{M}{2}\right) $$

(14)

where *Bri* means that the cyclic step size of row *i*, and *Bcj* is the cyclic step number of column *j*. Here, *i* = 1,2,...,*M*, *j* = 1,2,...,*N*.

*Step* **3***.* Color image *I* is decomposed into *R*, *G*, *B* parts, and then *R*, *G*, *B* parts are converted into three matrices and the rows are shifted. The shift results *TR*1, *TG*1 and *TB*1 are obtained by the following rules. Assumption *x*_{i} > 0, let the row *i* of *R* would be moved to left and step number is *Bri*; otherwise, the row *i* of *R* would be moved to the right with step number *Bri*, where *i* = 1,2,...,*M*. The same rules are used as in the *G* and *B* channels.

*Step* **4.** The columns shift results *TR*, *TG* and *TB* are obtained as follows. When *y*_{j} > 0, the column *j* of *TR*1 would be moved up with the size of step is *Bcj*, or else the column *j* of *TR*1 would be moved down with the size of step is *Bcj*, where *j* = 1,2,...,*N*. The same rules are used as in the *TG*1 and *TB*1.

#### DNA sequence operation

The pixel values are diffused according to DNA operations and include addition and complementary operations. Specific steps are as follows.

*Step* **1.** The *M* × 8 *N* binary matrices *R*, *G* and *B* are obtained by *TR*, *TG* and *TB*. Then the matrices *R*, *G* and *B* are encoded through the DNA encoding Rule α, and then the *M* × 4 *N* DNA matrix *S*1, *S*2 and *S*3 are obtained.

*Step* **2.** Setting the chaotic system initial values of *x*_{0}, *y*_{0}, *z*_{0}, *w*_{0} and getting chaotic sequences {*x*_{i}}*MN i = 1*, {*y*_{i}}*MN i = 1*, {*z*_{i}}*MN i = 1*, {*w*_{i}}*L i = 1* by iterating system (1) (*n* + *M* × *N*) times and discarding the former *n* values. Three sequences *k*1, *k*2 and *k*3 are obtained by

$$ \left\{\begin{array}{l}k1=\operatorname{mod}\left(\left\lfloor \left|{x}_i\right|\right\rfloor \right)\times {10}^{16},256\Big)\\ {}k2=\operatorname{mod}\left(\left\lfloor \left|{y}_i\right|\right\rfloor \right)\times {10}^{16},256\Big)\\ {}k3=\operatorname{mod}\left(\left\lfloor \left|{z}_i\right|\right\rfloor \right)\times {10}^{16},256\Big)\end{array}\right. $$

(15)

where *i* = 1,2,….,*MH*.

*Step* **3.** The sequence *k*1, *k*2 and *k*3 are transformed into binary matrix, and the matrixes are encoded according to the same DNA rule α to get three *M* × 4 *N* matrixes *K*1, *K*2 and *K*3.

*Step* **4.** According to the DNA complementary rule, the middle encryption result of DNA formulation matrix *C* = {*c*_{i}}*4MN i = 1* is obtained as follows:

$$ {\displaystyle \begin{array}{l}\mathrm{If}\ {\mathrm{c}}_{2i-2}=\mathrm{A},\mathrm{then}\ {\mathrm{c}}_{2i-1}={L}_{l1}\left({\mathrm{s}}_{2i-1}\right);\\ {}\mathrm{If}\ {\mathrm{c}}_{2i-2}=\mathrm{C},\mathrm{then}\ {\mathrm{c}}_{2i-1}={L}_{l2}\left({\mathrm{s}}_{2i-1}\right);\\ {}\mathrm{If}\ {\mathrm{c}}_{2i-2}=\mathrm{G},\mathrm{then}\ {\mathrm{c}}_{2i-1}={L}_{l3}\left({\mathrm{s}}_{2i-1}\right);\\ {}\mathrm{If}\ {\mathrm{c}}_{2i-2}=\mathrm{T},\mathrm{then}\ {\mathrm{c}}_{2i-1}={L}_{l4}\left({\mathrm{s}}_{2i-1}\right);\\ {}\mathrm{If}\ {\mathrm{c}}_{2i-1}=\mathrm{A},\mathrm{then}\ {\mathrm{c}}_{2i}={L}_{l5}\left({\mathrm{s}}_{2i}\right);\\ {}\mathrm{If}\ {\mathrm{c}}_{2i-1}=\mathrm{C},\mathrm{then}\ {\mathrm{c}}_{2i}={L}_{l6}\left({\mathrm{s}}_{2i}\right);\\ {}\mathrm{If}\ {\mathrm{c}}_{2i-1}=\mathrm{G},\mathrm{then}\ {\mathrm{c}}_{2i}={L}_{l7}\left({\mathrm{s}}_{2i}\right);\\ {}\mathrm{If}\ {\mathrm{c}}_{2i-1}=\mathrm{T},\mathrm{then}\ {c}_{2i}={L}_{l8}\left({\mathrm{s}}_{2i}\right).\end{array}} $$

*Step* **5.** The encrypted image of DNA sequence *D* = {*d*_{i}}*4MN i =* 1 is calculated by

$$ {d}_i={c}_i+K(i)+{d}_{i-1} $$

(16)

Here, *i* = 1,2,….,*MH*. “+” means that the DNA addition operation, and *d*_{0} = *c*_{4MN}. Three DNA matrices *D*1, *D*2 and *D*3 are obtained by Eq. (18).

*Step* **6.** The matrices *D*1, *D*2, and *D*3 are decoded by DNA Rule *β* and then recovering three binary formulations *C*1, *C*2 and *C*3. Finally, the encrypted image *C* by combination *C*1, *C*2 and *C*3 is obtained.

### Decryption algorithm

The decryption algorithm is a process of restoring the original image. First, the encryption image *C* is decomposed *C*1,*C*2 and *C*3, and then *C*1,*C*2 and *C*3 are encoded as matrices *D*1, *D*2 and *D*3 through DNA rule *β*, and then the middle encryption result of DNA formulation matrix *C* = {*c*_{i}}*4MH i = 1* is recovered as

$$ {c}_i={d}_i-K(i)-{d}_{i-1} $$

(17)

where *i* = 1,2,….,*MH*. “–” is the DNA subtraction, and *d*_{0} = *c*_{4MH}. The matrices *K*1, *K*2 and *K*3 are generated by doing Step 3 of the DNA sequence operation. Second, the image of DNA sequence matrices *S*1, *S*2 and *S*3 is recovered. The same iteration as Step 3 and Step 4 of pixel position scrambling is performed. Finally, the encrypted image is recovered.

### Simulation result

The color Lena image with the size of 256 × 256 is used for an algorithm simulation test, resulting in the Lena image as shown in Fig. 3a. The key *a* = 4, *b* = 0.01, *α* = 36, *β* = 20, *γ* = 3, *q* = 0.855, *ρ* = 2.67, *x*_{0} = 1, *y*_{0} = 0, *z*_{0} = 1, *w*_{0} = 0, *m* = 1000, *n* = 5000, *c*_{0} = *A*, *α* = 1, *β* = 3. The encrypted Lena image can be obtained as in Fig. 3b and the corresponding decryption image as shown in Fig. 3c.