Dec 29, 2020
Thanks for a clear explanation with lots of examples. Can we call a function from within the GradientTape()'s scope?
```python
W1 = tf.Variable(tf.random.normal((f,C)))
b1 = tf.Variable(tf.random.normal((C,)))
def forward_pass(X):
Z1 = X_train @ W1 + b1
A1 = tf.nn.softmax(Z1)
y_hat = A1
return y_hat
for i in range(epochs):
with tf.GradientTape() as tape:
y_hat = forward_pass(X_train)
# versus the inlined impl below
# Z1 = X_train @ W1 + b1
# A1 = tf.nn.softmax(Z1)
# y_hat = A1
J = - tf.reduce_sum( tf.math.log(y_hat) * y_train ) / m
```
I see different results b/w inlining and calling a function