Variable reuse in TensorFlow with example in Python

Hey Everyone,

I hope you all are doing well and enjoying the Learning Process.
In this article, we will be Understanding the Tensorflow variable and how to reuse it. Just like the variables in terms of Programming even TensorFlow has its own variable.
Now the question might arise how the programming variables vary from the Tensorflow variables?.
When it comes to TensorFlow the Tensorflow variable allows us to share variables in the program in a more robust way.
We will be understanding and executing the TensorFlow variable and its Reuse.

Let’s dive into the Explanation

We want to convert the Total Marks into the CGPA (Cumulative Grade Point Average) for 3 Students.
The Name of 3 Students are:

1. Sahil
2. Kartik
3. Kamlesh

Each student will enter their total marks out of let’s say 800 and it will be converting it into CGPA (out of 10).

Code and Explanation start here:

The module which we will be working on is Tensorflow.

import tensorflow.compat.v1 as tf
print("Tensorflow version 1 has been Imported")

Output:

 Tensorflow version 1 has been Imported

To disable the Version 2 behavior in the Tensorflow Program we can type the following:

tf.disable_v2_behavior()
print("Version 2 has been Disabled")

Output:

Version 2 has been Disabled

Let us define a function that will convert the Total Marks into CGPA which will have TensorFlow Variables. The datatype

def marks_to_cgpa(marks):
    """Convert Marks into CGPA
    """
    with tf.variable_scope("CGPA") as scope:
        total_marks = tf.get_variable("total_marks", (), dtype=tf.float32,
                              initializer=tf.constant_initializer(800.0))  # 0.05%
        total_cgpa = tf.get_variable("cgpa", (), dtype=tf.float32,
                                   initializer=tf.constant_initializer(10.0))  # $10
    student_marks = marks / total_marks * total_cgpa
    return student_marks
print("marks_to_cgpa function has been created")

Output:

marks_to_cgpa function has been created

CGPA Formula:

CGPA(Cumulative Grade Point Average)  =  Marks of Student  * 10
                                                                 ———————————–
                                                                  800

Let’s create a TensorFlow Global Variable initialization function, run the tensor (i.e. Sahil) and look at the Sahil’s CGPA.

def main():
    #Sahil marks is entered in the Function marks_to_cgpa
    sahil = marks_to_cgpa(450)

    init = tf.global_variables_initializer()  # Setting up the operator to assign all init values to variables

    with tf.Session() as sess:
        sess.run(init)  # Actually assign initial value to variables
        cgpa = sess.run(sahil) #Run the tensor name sahil
        print("CGPA of Sahil: %f" % (cgpa)) #After running the session we will get the CGPA of Sahil
if __name__ == "__main__":
    main()

Output:

CGPA of Sahil: 5.625000

Now, Looking at the below code we may conclude that the following will return CGPA of Kartik and Kamlesh.
Let run this code.

#CGPA for Kartik and Kamlesh
kartik = marks_to_cgpa(650)
print("CGPA of Kartik: %f" % (kartik))
kamlesh = marks_to_cgpa(780)
print("CGPA of Kamlesh: %f" % (kamlesh))

Output:

ValueError: Variable CGPA/total_marks already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:

  File "C:\Users\NIlesh\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1751, in __init__
    self._traceback = tf_stack.extract_stack()
  File "C:\Users\NIlesh\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3429, in _create_op_internal
    op_def=op_def)
  File "C:\Users\NIlesh\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3360, in create_op
    attrs, op_def, compute_device)
  File "C:\Users\NIlesh\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "C:\Users\NIlesh\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_core\python\framework\op_def_library.py", line 793, in _apply_op_helper
    op_def=op_def)

It is an Error because the Tensorflow Variable by default doesn’t allow to reuse its Variable.
To Execute the above set of code we have to set reuse as True (i.e.reuse = True). In the variable_scope of the marks_to_cgpa, we will add a parameter reuse = True in the variable_scope of the main().

The Function should look similar to the following code:

def marks_to_cgpa(marks):
    """Convert Marks into CGPA
    """
    with tf.variable_scope("CGPA", reuse=True) as scope:     # we have added a parameter reuse = True
        total_marks = tf.get_variable("total_marks", (), dtype=tf.float32,
                              initializer=tf.constant_initializer(800.0))  # 0.05%
        total_cgpa = tf.get_variable("cgpa", (), dtype=tf.float32,
                                   initializer=tf.constant_initializer(10.0))  # $10
    student_marks = marks / total_marks * total_cgpa
    return student_marks

Another issue with the set of code is that the function marks_to_cgpa returns a tensor and not the Real Number.
And hence we won’t be able to print the CGPA of Kartik and Kamlesh.

Let us Resolve the Errors and rewrite the code.

#CGPA for Kartik and Kamlesh

def main_new():
    init = tf.global_variables_initializer()  # Set up operator to assign all init values to variables
    with tf.Session() as sess:
        sess.run(init_op)  # Actually assign initial value to variables
        kartik = marks_to_cgpa(650)
        kamlesh = marks_to_cgpa(780)
        kartik_cgpa = sess.run(kartik)
        print("CGPA of Kartik: %f" % (kartik_cgpa))
        
        kamlesh_cgpa = sess.run(kamlesh)
        print("CGPA of Kamlesh: %f" %(kamlesh_cgpa))
if __name__ == "__main__":
    main_new()

Output:

CGPA of Kartik: 8.125000
CGPA of Kamlesh: 9.750000

Now, The  Problem is if reuse = True, we go into reuse mode for this scope as well as all sub-scopes.
And if reuse = tf.AUTO_REUSE, we create variables if they do not exist.

When you terminate the program and you run the code again while running main() the function which is for getting Sahil’s CGPA will generate an Error.

Output:

ValueError: Variable CGPA/total_marks does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?

To create a variable and reuse it when the variable does not exist we will type reuse = tf.AUTO_REUSE in the main() function.

I hope till here you had a great Journey.
I Highly recommend my reader to enter their total marks into the marks_to_cgpa and look at their result.

TensorFlow 2.0.0 Documentation: CLICK HERE