ISSN: 1992-8645 ESTABLISHING STRUCTURE FOR ARTIFICIAL .

Journal of Theoretical and Applied Information Technology10th March 2013. Vol. 49 No.1 2005 - 2013 JATIT & LLS. All rights reserved.ISSN: 1992-8645www.jatit.orgE-ISSN: 1817-3195ESTABLISHING STRUCTURE FOR ARTIFICIAL NEURALNETWORKS BASED-ON FRACTAL11Yang Zong-changSchool of Information and Electronical Engineering,Hunan University of Science and Technology, Xiangtan 411201, ChinaE-mail: 1yzc233@163.comABSTRACTThe artificial neural network (ANN) is a widely used mathematical model composed of interconnectedsimple artificial neurons, which has been applied in a variety of applications. However, how to determinenumber of neurons in the hidden layers is an important part of deciding overall neural network architecture.Many rule-of-thumb methods for determining the appropriate number of neurons in the hidden layers aresuggested. In this study, to the puzzling problem of establishing structure for the Artificial Neural Networks(ANN), from a microscopical view, two concepts called the fractal dimension of connection complexity(FDCC) and the fractal dimension of the expectation complexity (FDEC) are introduced. Then a criterionreference for establishing ANN structure based on the two proposed concepts is presented that, the FDCCmight not be lower than its (FDEC), and when FDCC is equal or approximate to FDEC, the ANN structuremight be an optimal one. The proposed criterion is examined with good results.Keywords: Artificial Neural Networks, Structure Establishing, Connection Complexity, Fractal Dimension.1.INTRODUCTIONAn artificial neural network (ANN), also called aneural network, is a widely used mathematicalmodel composed of an interconnected group ofsimple artificial neurons that also called nodes,neuroses, processing elements or units, areconnected together to form a network withmimicking a biological neural network. Artificialneural network (ANN) uses a connectionistapproach to computation in processing information,and is used with algorithms designed to change thestrength of the connections in the network to yield adesired signal flow. In most cases, an artificialneural network can be seen as an adaptive systemthat alters its structural weights during a learningstep. Artificial neural network is widely used tomodel complex relationships between its inputs andoutputs, and complex global behavior can bedetermined by the connections between itsprocessing elements and element parameters in thenetwork.Since its renaissance in early 1980s, artificialneural networks (ANN) research has received agreat deal of attention from the science andtechnology circles over the world [1-7]. Until now,besides so much attention has been given ANN, ithas also been reported fairly good performances forits nonlinear learning capability [1-7]. However, todetermine number of neurons in hidden layers [818] is a very important part of deciding overallneural network architecture for many practicalproblems employing neural networks [19-20].However, how to determine its structure especiallyof the hidden layer is a puzzling problem. Manymethods [8-18] for determining the appropriatenumber of neurons to use in the hidden layers areintroduced with varied degrees of success, such as,a method for estimating the number of hiddenneurons based on decision-tree algorithm in [9], anetwork structure equation by error function in[11], and one hidden layer train algorithm methodon energy space approaching strategy in [12], andan algorithm using an incremental trainingprocedure in [15], and some guidelines based on ageometrical interpretation of the multilayerperceptron (MLP) for selecting the architecture ofthe MLP in [16], and employing the singular vectordecomposition to estimate the number of hiddenneurons in a feed-forward neural network in [17],and some rule-of-thumb methods in [18]. Amongthe proposed solutions for this problem, some eitherfocus on the special training procedures that needsa large amount of operations and inconvenient forengineering applicability, or rule-of-thumb methodsthat short of generality.342

Journal of Theoretical and Applied Information Technology10th March 2013. Vol. 49 No.1 2005 - 2013 JATIT & LLS. All rights reserved.ISSN: 1992-8645www.jatit.orgA fractal can be view as a mathematical set,which usually has a fractal dimension exceeding itstopological dimension and may be one fractiondimension falling between two integers. Fractalsare typically self-similar patterns, where selfsimilar indicates that fractals may be exactly thesame measured at various scales.Addressing the important and puzzling problemthat to determine number of neurons in the hiddenlayers when deciding overall neural networkarchitecture, from a more macroscopical view, afractal-based approach is investigated in the study.2.FRACTAL AND FRACTAL DIMENSIONIn the well-known problem of the length ofBritish coastline, the author of the paper [21]-Mandelbrot discussed the research published byRichardson. Richardson had observed and foundthe famous formula as follows:L( r ) Kr1 D fE-ISSN: 1817-3195layer of 2n 1 neurons and m outputs, its structure,namely (n, 2n 1,m). The case is under the idealcondition for I [0, 1].From the Mapping Neural Network ExistenceTheorem [22], it indicates the inherent property ofmapping of an ANN [23, 24].Suppose an ANN with N inputs and M outputs,by ignoring the hidden layers and specificstructures, we can get a simplified topologicalstructure of the neural network. In the simplifiedtopological structure, with only 2 layers that theinput layer and output layer is considered, itsexpectation implementation function can be seen asone kind of “mapping” function and illustrated inFig.1, where AN denotes the input layer/unit and AMdoes the output layer/unit,ΑN AMMapping(1)Deep meaning for the exponent Df was notspecified by Richardson. In the paper of [21],Mandelbrot discussed self-similar curves, whichhave fractional dimensions between 1 and 2. Thisintroduced concept provides a new vision fordescribing many objects around us that havestructure on many sizes, whose normal examplesinclude coastlines, plant distributions and rivers,architecture, etc.By taking logarithm to Eq.(1) and makingnecessary mathematical operations, we get,D f log( L( r )) / log(1 / r )Fig. 1 Simplified topological structure of neuralnetworksThe fractal dimension is a mathematical concept,which measures the geometrical complexity of anobject. In follows, we try to propose twoconceptions based on the fractal dimension forANN:Definition1: we define ΩE as the called“Expectation Complexity” of the neural network inthe simplified topological structure (Fig.1) byΩ E (1 ρ E k ) S E (k 0)(2)(3)S N Size( AN ) denotes size ofand S M Size( AM ) for the outputSimply speaking, fractals are statistically selfsimilar. Where, self-similar means that fractals maybe exactly the same measured at various scales.Where,inputlayerlayer.Inspired, we present a fractal-based solution fordetermining number of neurons in the hidden layersof ANN in the following section.The ratio3. TWO CONCEPTS OF FRACTALDIMENSION FOR ANNρ E Max ( S N , SM ) / Min( S N , S M )indicates its mapping complexity that is expected tobeimplemented,andS E S( N , M ) Size( AN , AM ) size of both theinput and output layers, denotes its structuralcomplexity, and the user-defined parameter k 0.The Mapping Neural Network ExistenceTheorem[22] states that, given any continuousfunction, Φ:IN RM ,Y Φ(X ) ( I [0,1] ), where Xand Y are vectors with n and m componentsrespectively. This function can be implemented bya 3-layer neural network with n inputs, one hidden-In the simplified topological structure (Fig.1), theExpectation Complexity ΩE indicates ameasurement for the ANN with the 2 layers in thesimplified topological structure considered. Wefetch its scale of measurement (γE) by1/ γ E (1 1/ ρ E ) , with consideration of343

Journal of Theoretical and Applied Information Technology10th March 2013. Vol. 49 No.1 2005 - 2013 JATIT & LLS. All rights reserved.ISSN: 1992-8645possibility oftaken γ Ewww.jatit.orgE-ISSN: 1817-3195ρ E 1 that log(γ E ) 0 ifΩC {Ω1, Ω2 ,., Ωl 1} ρE .(5)In the case, the connection complexity ΩC ismeasured on l 2 layers thatANN ( AN , H 1, H 2, ., H l , AM ) , contains l 1Then we define a called “Fractal Dimension ofExpectation Complexity” by,DE Log (Ω E ) / Log (1/ γ E ) (4)connected sub-structures (Fig.3).According to Eq.(2), we have,The typical structure of multi-layer completelyconnected neural networks consists of the inputlayer, hidden layers and output layer (Fig. 2).Ω i (1 ρ i k ) S ( H i , H i 1 )Where,Input Layer Hidden Layer Output LayerHXY(6)(1 i l 1)ρi size( H j )( H j Ωi ) /( N M ) ，,S( H i , Hi 1 ) size( H i , H i 1 ) .Finally, we define DC as the called “FractalDimension of Connection Complexity”, which issum of the fractal dimensions on all its substructures:l 1Dc log(Ωi ) / log(1/ γ i )Fig. 2 Structure of multi-layer neural networksWhere, in Eq.(7), fetchSuppose N inputs in the input layer (unit), Moutputs in the out layer (unit), and l hidden layers(units) H i (i 1.l ) . There is l 2 layers in thestructure of ANN,. Hl , AMΩl 1)Fig. 3 Topological structure of Multi-layer neuralnetworkIn Fig.3, it shows that the topological structure hasl 1 connected sub fractal structures from (AN, H1)to (H1, AM). Then we define its called ConnectionComplexity as follows,Definition2: Define ΩC called “ConnectionComplexity” for the neural network with multilayer connected topological structure (Fig.3), andthe connection complexity of each sub-structure asΩi (1 i l 1),by1/ γ i (1 1/ ρ i )log(γ i ) 0 if taken γ i ρ i .4.Ll 2 size (AM) M, and so on.AN ( H1 . .Ω1,γiwith consideration of possibility of ρ i 1 that( AN , H1, H 2, ., H l , AM ) (Fig.3).Number of neurons in each layeris, ( L1, L2, ., Ll 1 , Ll 2 ) where L1 size (AN) N,(7)i NING OF ANN PARAMETERSBased on the presented two concepts of fractaldimension for ANN in Section2, namely the fractaldimension of expectation complexity (DE)((Eq.(4))and fractal dimension of connection complexity(DC)(Eq.(7)) , we propose a criterion reference forestablishing ANN structure as follows,To establish ANN structures, the fractal dimensionof connection complexity (Dc) might not be smallerthan its fractal dimension of expectation complexity(DE),Dc DE(8)When the fractal dimension of connectioncomplexity (Dc) is equal or approximate to itsfractal dimension of expectation complexity (DE),i.e., Dc DE or Dc DE, the established structuremight be an optimal one.344