A New Learning Control System for Basketball Free Throws Based on Real Time Video Image Processing and Biofeedback

Shooting free throws plays an important role in basketball. The major problem in performing a correct free throw seems to be inappropriate training. Training is performed offline and it is often not that persistent. The aim of this paper is to consciously modify and control the free throw using biofeedback. Elbow and shoulder dynamics are calculated by an image processing technique equipped with a video image acquisition system. The proposed setup in this paper, named learning control system, is able to quantify and provide feedback of the above parameters in real time as audio signals. Therefore, it yielded to performing a correct learning and conscious control of shooting. Experimental results showed improvements in the free throw shooting style including shot pocket and locked position. The mean values of elbow and shoulder angles were controlled approximately on 89 and 26, for shot pocket and also these angles were tuned approximately on 180 and 47 respectively for the locked position (closed to the desired pattern of the free throw based on valid FIBA references). Not only the mean values enhanced but also the standard deviations of these angles decreased meaningfully, which shows shooting style convergence and uniformity. Also, in training conditions, the average percentage of making successful free throws increased from about 64% to even 87% after using this setup and in competition conditions the average percentage of successful free throws enhanced about 20%, although using the learning control system may not be the only reason for these outcomes. The proposed system is easy to use, inexpensive, portable and real time applicable. Keywords-learning; control; free throw; basketball; image processing; biofeedback; shot pocket; locked position


INTRODUCTION
The shot (including 3-point shot, 2-point shot and free throw) is one of the most important techniques in basketball.These shots have approximately the same dynamics (especially of hands) whereas the easiest way to score is considered the free throw [1].Based on the new basketball rules and the changes in the athlete's physical conditions from ectomorphy to mesomorphy, contacts and therefore the probability of fouls in a match are increased.On the other hand, increasing the distance of 3-point shots [2] should result in a decreased probability of distant scoring and to an increase of in game playing and to an increase to the number of free throws per game.A free throw is a shot taken from a certain distance while the game has paused as result to a foul.Therefore, a high success percentage is expected (notable higher compared to 3point and 2-point shots).However, missed free throws often decide the final outcome of a match.Some examples are shown in Table I (from matches from the 2008 FIBA Under-18 Asian Championship).
Applying sports engineering methods has resulted in more effective performance in several sports including basketball [4].Computational methods using biofeedback have the potential to improve the free throw dynamics.Previous investigations on free throw applied 2D analysis techniques.The study of throw techniques from three different distances led to the conclusion that all of them used the technique of coordination of pushing [5].It was discovered that good throws presented larger shoulder angle during pushing the ball [6].These studies mainly focused on identifying kinematic variables related to successful throws and ignored the way they behave when the distance from the basket changes.In [7] authors studied the relationship between selected biomechanical free throw parameters on a sample of basketball players with various capacities.Successful free throws are found not to be contingent upon speed, angle or height of the throw and they mostly depend on the precision of the player [8].In [9], authors compared male and female basketball players in terms of 2 and 3-point jump shots.They reported that joint angles at release have no significant changes for the two groups.In [10] authors found that female players had larger joint range from the time they crouch to the time of release (77.5 o ) and men had smaller (60.7 o ).Authors in [11] found that the angle of releasing the ball for 3 distances (short range, medium range and long range) from the basket in basketball shooting, is similar and approximately around 49.50±7.13o .In [12], authors studied the relation of kinematics of shooting, the distance to the basket and player position.They found out that as the distance from the basket increases, players have higher angular velocity because they have higher angular velocities of shoulder flexion and elbow extension, and also the speed of the center of mass in the direction through the basket is higher.In [13], authors assessed the relation between distance, outcome and performance of basketball jump shot, and reported that in close distance, the release angle is higher than in intermediate distance (78.92 o to 65.60 o ).Authors in [14] used a model for basketball player arms to estimate angles, velocity and angular velocity of joints at release condition.Previous works so far have not been combined with computational feedback measurements and instrumentations to amend and control a player's free throw technique.By this feedback, a correct reference signal as a desired trajectory would be produced and sent to the athlete in real time.After primary training epochs, the athlete would be able to control the movement around the desired trajectory.In fact, measurement (quantitation) and biofeedback have a great role in motor control [15].Biofeedback has wide applications in sports and sports engineering [16,17].Most of them are in controlling athletes' psychological parameters [18].The biofeedback approaches in sports are commonly divided in five groups [19] which are muscular biofeedback (EMG) [17,20], thermal biofeedback [21], electrical dermal activity (EDA or GSR) biofeedback [22], EEG biofeedback [22,23] and cardiovascular biofeedback [24].These approaches are used to improve the athlete's performance [25].Biofeedback has also applications in sports learning, training and rehabilitation [26,27] but it actually has focused on psychological problems.
Our study is intended in accordance to the biofeedback approach focusing on movement control, so the free throw control system can be categorized in the biofeedback field [16,28].This study intends to introduce a new learning control system as training assistance, while trying to show a conjunction between control engineering, video image processing, biofeedback, neuromuscular control system and sports.First, the shooting technique is analyzed by using video image acquisition and processing methods.Next, the closed loop system is applied to quantify and feedback the movement parameters as audio signals to subjects via their auditory system.Finally, correct learning and conscious shooting control are assessed through the experiments.

A. Participants
Participants were professional and amateur basketball players of a university basketball team.All participants were at least 19 years old and of varying basketball skill.A set of experiments was made on some famous and national athletes in order to extract their movement patterns and compare it with the desired reference pattern.

B. The Free Throw Technique
Even though participants had basketball-related experience, instructions regarding the correct free throw technique were given before the experiments.They should shot in a plane which is vertical to the ground and is peripheral to the assumed line between eye and the center of the basket (shooting plane).The standard of correct free throw technique was derived from [1,29].They were also asked to supervise their hand visually in order not to exit from the shooting plane.However, some effective parameters in shooting can't be observed visually.One of the most important problems corresponding to shooting is its initial condition named "shot pocket" which consists of the state of elbow and shoulder angles.The elbow and shoulder angles should be approximately 90 o and 25 o , respectively.Generally, if the SP is lower, the ball will be thrown at a lower height.Therefore, there will be less chance of making the shot which for a defender will be easier to block or control it.Also if a player shoots with the ball in a low pocket, it could be seen that the elbow is inside (medial) or outside (lateral) the center of the ball (usually for younger players with weak arms and a lack of upper body strength).If the elbow is not directly under the center of the ball then this creates inconsistent direction [1].If the SP is high, the arc and height of the throw will be larger and therefore there will be more chance of making the shot, it will be more difficult to defend but it requires more force.Moreover, when the SP is too high the hand and the ball may exit from the adequate field of vision [1].Another important matter in shooting is the final state or "locked position" (LP).Τhe final angles of elbow and shoulder must be 180 o and 45 o , respectively [1].This occurs simultaneously by the releasing of the ball.In other words, the elbow is completely flat and the arm moves up approximately 20 o to 25 o .We use this paradigm as a reference or desired trajectory.

C. The Learning Control System
This system, firstly measures the effective parameters in shooting e.g. the angles of elbow and shoulder and their dynamics.Then simultaneously it sends their feedback to the athlete.These parameters are not recognized by the eye and basically they are not presented to brain consciously and accurately.This system tries to solve it by creating new feedback loops.Therefore, we can consider this instrument as a new biofeedback system.Figure 1(a) shows the block diagram of this system which will be introduced in parts.Figure 1(b) shows the block diagram of athlete's neuromuscular control system.θ 1 and θ 2 are elbow and shoulder angles, C 1 and C 2 are neural motor commands, they show the relations between external tools and internal subsystems in the athlete's body and f 1 and f 2 are frequencies of generated sounds.

 Camera
The selected camera is a simple digital camera with PAL System, interlaced imaging, fire-wire output for IEEE 1394 computer image acquisition port and AVI image format with a resolution of 520×780 pixels.By the interlaced imaging system, the camera divides every 40 ms period into two 20 ms segments.In each segment, it captures half of the image as alternate rows, which means two frames instead of one [30].The image separation in two parts is selection of image matrix rows as alternation.By this method and in case that the marker size is chosen appropriately, it is possible to achieve twice the frame rate (50 fps) resulting in enhanced accuracy.Hence for a sampling period of 20 ms and about 180 ms elbow movement duration, the correct sampling rate has been considered sufficiently in the sense of the Nyquist criterion [31].
 The Markers Choosing of markers depends on many factors such as the camera type, the motion dynamic and the recording environment.It should be emphasized that this paper introduces an instrument which is very simple, accurate and easy to use.So these markers should be very lightweight, with homogeneous color intensity and easy to distinguish.In this study, the researchers have used active markers.

 Image import
The computer used in this project is a 2 GHz Core 2 Due CPU Laptop with 2 GB RAM.Since this computer has an IEEE 1394 video in port and the related hardware, it can satisfy the conditions of the desired system in order to acquire video images with minimum delay.Because of the sampling rate of 25 fps and the AVI video image format (576×720 pixels per image or 576×720×3/1024=1215 Kb=1.1865Mb), the amount of 29.6625 (1.1865×25) Mb per second is needed to be transmitted.So, using a minimum signaling rate of IEEE 1394 devices (98.304Mbps for S100 standard), we can import the video images to the computer without any delay.The standard interlaced technique is to transmit the picture in two pieces (or fields), called 2:1 interlaced scanning [30].In the interlaced scan in each period scanning (40 ms for 25 fps) two images with 20 ms interval are taken instead of one.So, based on interlaced imaging technique 50 images per second will be ready for the video image processing subsystem.

2)
Video Image processing The processing was performed on images with size of 288x20 pixels.The first part of the algorithm written in MATLAB, determines the location of the markers based on color image processing [32] and circle detection using Hough transform (in offline mode for the algorithm validation) [33].All marker coordinates are defined toward the origin which is placed on the northwest corner of the image (Figure 2).The coordinate axes X and Y are represented for the width and height of the image, the vectors m 1 , m 2 , and m 3 are the marker's vectors and vectors b and s are representing the effective arms in the movement which are arm and forearm vectors respectively.
where θ 1 and θ 2 are the angles of elbow and shoulder respectively and i x is the unit vector along X-axis.
The angular velocity and acceleration (for the time t i ) are calculated by the use of the approximation equations: where ω and α are angular velocity and acceleration respectively.The measurement system has a resolution of 0.01 o and therefore 0.005 o error.
Fig. 2. The Marker's position on the hand which is identified by three vectors in an assumed plane.

3) Conversion of Information to Audio Signals via Wireless Communication
The elbow and shoulder angle information and their acceleration were transformed into audio signals.We used a beep alarm with a smooth frequency (like 500 Hz) as discontinuous periodic audio signals.The period of this signal is tuned related to the angle (with a fixed duty cycle).Equations ( 4)-( 8) represent the conversion of the motor quantities to proportional audio signals: ( ) where h 1 and h 2 are pulse train and beep signals respectively, f b is the proper frequency for generating the beep alarm, T i is the period of the pulse train signal and is proportional to joint angle or Δθ i which is the difference between θ i and the desired angle (in other words T i is a function of θ i ), d is the duty cycle of pulse train signal and it must be less than the imaging period and p(t) is the final produced audio signal which is the angle audio feedback.Also, τ 1 , τ 2 are the agonist and antagonist muscle torques on hand arms, τ is the consequent torque, J is the inertia of the effective hand arm and α is the angular acceleration (see ( 3)).Moreover, a forced audio feedback can be made through the same method based on the second rule of Newton.The torque (force) is relative to the angular acceleration.Forced audio feedbacks can be based on appropriate musical notes around the desired force.The produced audio signals were broadcasted through a wireless headphone like FM modulator.

4)
The role of neuromuscular control system Figure 1(b) shows the block diagram of neuromuscular control system generally in the learning control system and especially in the shooting.Figure 3 shows a model of neuromuscular control system in voluntary movements based on feedback error learning approach presented in [32,34].The block diagram represents the control system of voluntary movements considering the role of cerebellum as well as the cerebrum as the controllers of motion.According to this model, when the association cortex (ASSN CX) receives the idea of movement from limbic cortex, it sends the desired movement pattern as θ d (which is an expression of the state of limbs) to cerebellum and cerebrum's motor cortex.Here the motor commands 'T' related to torques which should be produced in muscles are calculated and then they will be applied by the muscles.The inner sensors namely muscle proprioceptor and joints such as muscle spindles, should measure the actual occurred behavior of limbs 'θ'.Therefore the feedback based movement control can be performed using error signal (

ltieswww.etasr.com Sarang et al.: A New Learning Control System for Basketball Free Throws Based on Real Time Video …
On the other hand, the cerebellum makes an inverse model to produce the desired commands.According to this model and some physiological facts, this section receives the desired trajectory 'θ d ' as an input and the motor commands resulted from the cerebrum's motor cortex, T fb (or, to be more precise, the motor command error (∆T)).T fb is used to correct the model by tuning the weights of neural networks of cerebellum in order to decrease the error.Finally, the feedback exits the circuit and the cerebellum learns the inverse model and the system is turned into a feedforward control system [34].In other words, after correct training and learning and finally gaining the skill, all shots, theoretically, must be successful.The most important matter in the neuromuscular control system effective in shooting is the occurrence of inaccurate unconscious feedbacks to the brain.In this study, based on physiological facts, we try to modify the existing feedbacks, quantify them, raise awareness about them and design more appropriate feedback paths by a new biofeedback technique.

A. Comparison of Elbow and Shoulder Angles Before and After Applying the Learning Control System
As shown in Figure 4, the elbow and shoulder angles are about 100 o and 27 o in SP deviating from the desired trajectory (especially in elbow) and about 180 o and 53 o in LP pocket again deviating from the desired trajectory (especially in shoulder).There are some variations even for professional athletes because of bad training.The SP and LP were corrected for all players by using the learning control system.Figure 5 displays the elbow and shoulder angles in a free throw by a participant before using the system and Figure 6 shows the elbow and shoulder angles after applying it.The elbow and shoulder angle in a free throw by a famous National player.
Before using the learning control system the elbow and shoulder angles were calculated as 85.14±11.320and 35.53±8.650respectively for SP and 175.45±7.540and 61.91±11.360respectively for LP.After using the learning control system, the values measured was 89.32±3.260and 26.12±2.430for SP and 180±000 and 46.72±2.650for LP, for elbow and shoulder angle respectively.These results clearly show the improvement also shown by the decrease in the mean error from the desired trajectory (about 4.1 and 9.4 degrees for SP and 4.5 and 15.2 for LP) and also from the decrease of the deviation from the mean value and consequently show the achieved uniformity in the shooting style (about 8.1 and 6.2 degrees for SP and 7.5 and 8.6 degrees for LP).The elbow and shoulder angles in a free throw by a participant before using learning control system.The elbow and shoulder angle in a free throw which was improved becauseof using the learning control system.

B. Assessment of Performance
We applied the learning control system to about 15 players of varying skill.The repetition number depended on the athlete's level.After necessary repetitions and drills using the setup, an improvement in athletes' performance was observable.Both of SP and LP were corrected towards the desired references.Also the percentage of successful free throws increased remarkably.Before using this system, in training conditions, the average percentage of making free throws was about 64.4%±8.23 which improved to 87.34%±3.46.Also 5 of 15 players were supervised during season matches.In those matches, the average performance was about 68.4%±3.71which showed about a 20% improvement from their previous statistics.

A. Study Findings
The study has numerically combined dynamic measurements of 15 basketball players with a new shooting control system to consciously amend the free throw technique during training.Their elbow and shoulder angles were compared before and after applying the learning control system.Both mean value and standard deviation of these angles improved significantly.After training with the learning control system, the elbow and shoulder angles were 89.32±3.26 and 26.12±2.43respectively for SP and 180±0 and 46.72±2.65 respectively for LP.Final mean values are close to the desired pattern (90 and 25 for SP and 180 and 45 for LP), demonstrating the system's accuracy.Also meaningful decrease of the standard deviation from 11.32 and 8.65 to3.26 and 2.43 for SP and from 7.54 and 11.36 to ±0 and 2.65 for LP show player shooting style uniformity and demonstrate the system's precision.It should be noted that the essential forces when shooting a free throw change based on angular acceleration and could be modified based on acceleration feedbacks.
Moreover, the players' performance was also considered.In training condition, the average success of 64.4% of total free throw shooting was improved to 87.34% and the standard deviation significantly decreased about 4.2% (from 7.23 to 3.26).Also in competition conditions the average performance showed about 20% improvement.Of course many parameters influence the result of a shot (e.g.mental concentration and precision, physical condition, fatigue level, opponent defense and other movement parameters), but especially in free throw most of them are absent or limited because of the static conditions of the shot.Moreover, by this system the theory of feedforward control of voluntary skillful movement seems to be confirmed.At first the desired movement pattern is formed in the brain.Then neuromuscular control system starts learning by making the correct inverse model of the movement.After sufficient training steps, the brain can control the free throw as an open loop system and after that the feedback is not necessary.Based on our knowledge this is the first time that an online, persistent and conscious modifying setup has been combined with dynamic measurements of athletes' shooting to enable amelioration of free throw as a new biofeedback approach.Despite the use of simple and cheap active markers, the common camcorders with an interlaced imaging system, the ordinary laptops and appropriate video image processing algorithms, overall improvement in free throw style was achieved.Additionally, in this regard we can say that our setup makes a strong connection between control engineering, real time video image processing, biofeedback and neuromuscular control systems.

B. Limitations and Future Trends
Assuming the hand moves in the shooting plane, 2D images were used whereas actually this movement is not quite 2D and it is needed to process 3D images via at least 2 cameras, though in this work some features (not actual angle) were defined, calculated and applied for all athletes using the same methodology.It is possible to generalize the results and apply them to other types of shooting like 2 and 3 point jump shots, because their dynamic is almost similar to the free throw.Besides, by defining and using additional movement features, it is possible to learn and control the shooting skill in a better way.Moreover, by making small changes, this system can be used for learning and controlling movements in other sports.

V. CONCLUSION
We have introduced a simple setup of combined dynamic measurements of basketball shots with a novel shooting control system based on new quantified feedback loops to consciously amend the free throw technique.Our system corrects and controls free throw SP and LP toward the desired references while variations in shooting style are decreased.Moreover, the learning control system influences the successful free throw while the variations in results decrease.The acceptable mean value and decreased standard deviation of both movement corrections (SP and LP) and player performance improvement show the accuracy and precision of the whole learning control system.The advantages of using a simple setup including active markers, common camcorders and a laptop as a processor for implementation of simple video image processing algorithms, resulted to an applied online closed loop learning control system.

Fig. 1 .
Fig. 1.The block diagram of learning control system: (a) The block diagram of shooting control system, (b) the block diagram of athlete's neuromuscular control system.

1 )
It can be clearly seen that the angles of elbow and shoulder are just the same as the angles between the vectors s  and b  , and between vector b  and X-axis.So we have:

Fig. 3 .
Fig. 3.The block diagram of movement control model consisting of the upper centers (cerebrum and cerebellum) and lower centers (spinal cord and muscles).
Fig. 4.The elbow and shoulder angle in a free throw by a famous National player.
Fig.5.The elbow and shoulder angles in a free throw by a participant before using learning control system.
Fig.6.The elbow and shoulder angle in a free throw which was improved becauseof using the learning control system.

TABLE I .
THREE UNDER-18 ASIAN CHAMPIONSHIP 2008 MATCHES:FINAL RESULTS AND FREE THROW STATISTICS..